Multi-level navigation monitoring and control

ABSTRACT

Multi-level navigation monitoring and control is provided. A system includes a lane marking manager determining a first boundary line, a second boundary line, and a centerline of a current lane of travel. The system also includes a confidence level determiner assigning a first confidence level to the first boundary line, a second confidence level to the second boundary line, and a third confidence level to the centerline. Further, the system includes a user interface outputting representations of the first boundary line, the second boundary line, and the centerline based, at least in part, on the first confidence level, the second confidence level, and the third confidence level.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part (CIP) of, and claims thepriority of, pending U.S. Non-Provisional patent application Ser. No.14/321,105 entitled “ENHANCED 3-DIMENSIONAL (3-D) NAVIGATION,” filed onJul. 1, 2014, which is a continuation-in-part (CIP) of pending U.S.Non-Provisional patent application Ser. No. 14/041,614 entitled“3-DIMENSIONAL (3-D) NAVIGATION”, filed on Sep. 30, 2013, which is acontinuation-in-part (CIP) of pending U.S. Non-Provisional patentapplication Ser. No. 13/832,918 entitled “VOLUMETRIC HEADS-UP DISPLAYWITH DYNAMIC FOCAL PLANE”, filed on Mar. 15, 2013. The entirety of theabove-noted applications are incorporated by reference herein.

BACKGROUND

When a vehicle is being driven, various information may be important tothe driver as well as the other vehicle occupants. Such information mayinclude maps and related mapping information, as well as otherinformation that may be useful to assist the driver/occupants to travelfrom place to place. From the perspective of a driver/occupant, theinformation may be difficult to obtain or may be cumbersome to use in amoving vehicle.

SUMMARY

This brief description is provided to introduce a selection of conceptsin a simplified form that are described below in the detaileddescription. This brief description is not intended to be an extensiveoverview of the claimed subject matter, identify key factors oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

An aspect relates to a system for multi-level navigation monitoring andcontrol. The system may include a memory and a processor that executescomputer executable components stored in the memory. The computerexecutable components may include a navigation manager generating aroute between a first location and a second location and a map displayeroutputting a map that includes the route and is indicative of distancesof one or more surfaces, objects, obstructions, or geometries in an areaaround the vehicle. Further, the computer executable components mayinclude a modification controller altering at least a portion of theroute based on an indication of a change received at a user interface ora sensor. The navigation manager regenerates the route between the firstlocation and the second location as a function of the indication of thechange.

Another aspect relates to a method for multi-level navigation monitoringand control. The method may include generating a route between a firstlocation and a second location based on a determination of a currentlocation of a vehicle and a received destination location. The methodmay also include outputting a map on a windshield of the vehicle. Themap may include segments of the route and may be transparent. A view ofreal world objects through the windshield might not be obstructed by thesegments. Further, the method may include altering at least one segmentof the route based on an indication of a change to the at least onesegment. The altering may include overriding an autonomous steeringinstruction of the vehicle.

Still another aspect relates to a system for navigation monitoring andcontrol of a vehicle. The system may include a memory and a processorthat executes computer executable components stored in the memory. Thecomputer executable components may include a lane marking managerdetermining a first boundary line, a second boundary line, and acenterline of a current lane of travel. The computer executablecomponents may also include a confidence level determiner assigning afirst confidence level to the first boundary line, a second confidencelevel to the second boundary line, and a third confidence level to thecenterline. Further, the computer executable components may include auser interface outputting representations of the first boundary line,the second boundary line, and the centerline based, at least in part, onthe first confidence level, the second confidence level, and the thirdconfidence level.

The following description and annexed drawings set forth certainillustrative aspects and implementations. These are indicative of but afew of the various ways in which one or more aspects are employed. Otheraspects, advantages, or novel features of the disclosure will becomeapparent from the following detailed description when considered inconjunction with the annexed drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the disclosure are understood from the following detaileddescription when read with the accompanying drawings. Elements,structures, etc. of the drawings may not necessarily be drawn to scale.Accordingly, the dimensions of the same may be arbitrarily increased orreduced for clarity of discussion, for example.

FIG. 1 is an illustration of an example schematic diagram of a vehicularheads-up display system according to one or more embodiments of thisdisclosure.

FIG. 2 is an illustration of an example schematic diagram of a vehiclein which a vehicular heads-up display system is provided according toone or more embodiments of this disclosure.

FIG. 3 is an illustration of an example side view of a vehicle and fourfocal planes on which graphic elements are projected by a vehicularheads-up display system according to one or more embodiments of thisdisclosure.

FIG. 4 is an illustration of an example view of a driver while driving avehicle, looking through a windshield of the vehicle, and exemplarygraphic elements projected by a vehicular heads-up display systemaccording to one or more embodiments of this disclosure.

FIG. 5 is an illustration of an example component diagram of a systemfor 3-D navigation according to one or more embodiments of thisdisclosure.

FIG. 6 is an illustration of an example flow diagram of a method for 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 7A is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 7B is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 8A is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 8B is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 9A is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 9B is an illustration of an example avatar for 3-D navigationaccording to one or more embodiments of this disclosure.

FIG. 10A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 10B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 11A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 11B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 12A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 12B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 13A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 13B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 14A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 14B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 15 is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 16A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 16B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 17A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 17B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 18A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 18B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 19A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 19B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 20A is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 20B is an illustration of an example scenario associated with 3-Dnavigation according to one or more embodiments of this disclosure.

FIG. 21 is an illustration of an example flow diagram of a method for3-D navigation according to one or more embodiments of this disclosure.

FIG. 22 is an illustration of an example, non-limiting system configuredfor multi-level driver monitoring and/or control according to one ormore embodiments of this disclosure.

FIG. 23A illustrates an example scenario related to interfacing with avehicle occupant according to one or more embodiments of thisdisclosure.

FIG. 23B illustrates another example scenario related to interfacingwith a vehicle occupant according to one or more embodiments of thisdisclosure.

FIG. 24A illustrates an example scenario related to allowing a vehicleoccupant to monitor and/or control navigation information according toone or more embodiments of this disclosure.

FIG. 24B illustrates another example scenario related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure.

FIG. 24C illustrates another example scenario related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure.

FIG. 24D illustrates a further example scenario related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure.

FIG. 25 illustrates example, non-limiting, indications of futuremaneuvers that may be utilized according to one or more embodiments ofthis disclosure.

FIG. 26 illustrates an example scenario related to providing visualguides for a current level of steering control according to one or moreembodiments of this disclosure.

FIG. 27 illustrates an example scenario related to an incorrectdetermination of a current level of steering control according to one ormore embodiments of this disclosure.

FIG. 28 illustrates an example scenario related to an output indicatinga confidence level associated with a current level of steering controlaccording to one or more embodiments of this disclosure.

FIG. 29 illustrates an example, non-limiting, method that providesmulti-level driver monitoring and control according to one or moreembodiments of this disclosure.

FIG. 30 illustrates an example, non-limiting method for navigationmonitoring and control of a vehicle according to one or more embodimentsof this disclosure.

FIG. 31 is an illustration of an example computer-readable medium orcomputer-readable device including processor-executable instructionsthat embody one or more of the provisions set forth herein according toone or more embodiments of this disclosure.

FIG. 32 is an illustration of an example computing environment where oneor more of the provisions set forth herein are implemented according toone or more embodiments of this disclosure.

DESCRIPTION

Embodiments or examples, illustrated in the drawings are disclosed belowusing specific language. It will nevertheless be understood that theembodiments or examples are not intended to be limiting. Any alterationsand modifications in the disclosed embodiments, and any furtherapplications of the principles disclosed in this document arecontemplated as would normally occur to one of ordinary skill in thepertinent art.

For one or more of the figures herein, one or more boundaries, such asboundary 116 of FIG. 2 or boundary 1030 of FIG. 10B, for example, aredrawn with different heights, widths, perimeters, aspect ratios, shapes,etc. relative to one another merely for illustrative purposes, and arenot necessarily drawn to scale. For example, dashed or dotted lines areused to represent different boundaries, thus, if the dashed and dottedlines were drawn on top of one another they would not be distinguishablein the figures. Therefore, the lines are drawn with different dimensionsor slightly apart from one another, in one or more of the figures, sothat the lines are distinguishable from one another. As another example,where a boundary is associated with an irregular shape, the boundary,such as a box drawn with a dashed line, dotted lined, etc., does notnecessarily encompass an entire component in one or more instances.Conversely, a drawn box does not necessarily encompass merely anassociated component, in one or more instances, but may encompass aportion of one or more other components as well.

Graphic elements visually placed on environmental elements in the directview of a driver by a vehicular heads-up display (HUD) device are oftenreferred to as contact-analog or conformal augmented reality graphicelements. Successfully presenting contact-analog augmented realitygraphic elements to the driver of a vehicle may depend on the ability ofthe vehicular HUD device to reproduce depth cues. These depth cues mayinclude accommodation and vergence. Accommodation is a depth cue wherethe muscles in the eye actively change the optical power to change focusat different distances. Vergence is the simultaneous or concurrentinward rotation of the eyes towards each other to maintain a singlebinocular image when viewing an object.

Although examples described herein may refer to a driver of a vehicle,graphic elements may be projected, provided, rendered, etc. within viewof one or more other occupants of a vehicle, such as passengers, etc. Inaddition, these examples are not intended to be limiting, and are merelydisclosed to illustrate one or more exemplary aspects of thisdisclosure.

When a HUD device displays a graphic element on a windshield of avehicle, accommodation may cause the human eye to shift betweenenvironmental elements and information displayed by the HUD device.Vergence causes the eyes to converge to points beyond the windshieldinto the environment, which may lead to the appearance of a double imageof the HUD graphic element displayed on the windshield. Accordingly, torender contact-analog augmented reality graphic elements with accuratelyreproduced depth cues, graphic elements should be rendered on the samespace as the real environment (e.g., at corresponding focal planes),rather than on the windshield of the vehicle.

A vehicle HUD device for displaying graphic elements in view of a driverof a vehicle while the driver views an environment through a windshieldis provided. The HUD device may include one or more projectors thatproject a graphic element on a frontal focal plane in view of the driverwhile the driver views the environment through the windshield.Additionally, the HUD device may include one or more projectors thatproject a graphic element on a ground-parallel focal plane in view ofthe driver while the driver views the environment through thewindshield. The projector that projects the graphic element on thefrontal focal plane may be mounted on an actuator that linearly movesthe projector to cause the frontal focal plane to move in a direction ofa line-of-sight of the driver. The projector that projects theground-parallel focal plane may be fixedly arranged such that theground-parallel focal plane is static.

FIG. 1 is an illustration of an example schematic diagram of a vehicularheads-up display system according to one or more embodiments of thisdisclosure. A vehicular volumetric heads-up display system 100 (“HUDsystem 100”) or (“HUD component 100”) capable of rendering volumetriccontact-analog augmented reality graphic elements is illustrated. Thegraphic elements may include 3-dimensional or “3-D” graphic elementsrendered into the same space as the real environment with accuratelyreproduced depth cues. The HUD system 100 may include a vehicle heads-updisplay device 102 (“HUD device 102”) and a controller 104 (or“controller component 104”). FIG. 2 is an illustration of an exampleschematic diagram of a vehicle in which a vehicular heads-up displaysystem is provided according to one or more embodiments of thisdisclosure. As illustrated, the HUD system 100 may be provided in avehicle 106, which may include a driver seat 108, a dashboard enclosure110, and a windshield 112.

The configuration of the vehicle 106, with respect to the relativepositioning of the driver seat 108, dashboard enclosure 110, andwindshield 112, for example, may be conventional. To accommodate theherein-described HUD system 100, the dashboard enclosure 110 defines ahousing space in which the HUD system 100 is housed. Further, thedashboard enclosure 110 has a HUD exit aperture 114 defined through anupper surface thereof. The HUD system 100 housed in the dashboardenclosure 110 may project graphic elements, such as contact-analogaugmented reality graphic elements, through the HUD exit aperture 114 tothe windshield 112, which may be used as a display screen for the HUDsystem 100. As described in further detail below, the augmented realitygraphic elements may be rendered to the driver as if in the same spaceas the real environment.

A driver of the vehicle 106 drives the vehicle 106 while seated in thedriver seat 108. Accordingly, the driver may be positionally constrainedto a seating position on the driver seat 108 within the vehicle 106. Inview of this positional constraint, the HUD system 100 may be designedusing an assumption that the driver's view originates from an eye box116 within the vehicle. The eye box 116 may be considered to include aregion of an interior of the vehicle 106 where the driver's eyes aresituated while the driver is seated in the driver seat 108.

The eye box 116 may be sized to encompass head positions of the driverregardless of a position and posture of the driver seat 108, or the HUDsystem 100 may detect the position and posture of the driver seat 108,and to adjust a position and size of the eye box 116 based thereon. Inone or more embodiments, the HUD system 100 may be designed assuming theeye box 116 has a fixed size and is in a fixed position. For example,the eye box may have the following dimensions: 20 cm×10 cm×10 cm. TheHUD system 100 may present the contact-analog augmented reality graphicelements to the driver when the driver's eyes are within the eye box 116and the driver is facing/looking in a forward direction through thewindshield 112 of the vehicle 106. Although the eye box 116 of FIG. 2 isillustrated for the driver of the vehicle 106, the eye box 116 mayinclude one or more other occupants of the vehicle. In one or moreembodiments, one or more additional eye boxes or HUD devices may beprovided for passengers or other occupants, for example.

The HUD device 102 may display one or more graphic elements in view ofthe driver of the vehicle 106 while the driver views an environmentthrough the windshield 112 of the vehicle 106. Graphic or environmentalelements viewed by the driver through the windshield 112 while thedriver's eyes are in the eye box 116 and the driver is facing/looking inthe forward direction through the windshield 112 may be considered to bein view of the driver. As used herein, the view of the driver of thevehicle 106 while the driver views an environment through the windshield112 of the vehicle 106 is intended to include an area viewed through thewindshield 112, excluding dashboard displays located within the vehicle106. The HUD device 102 may present the graphic elements such that thedriver may view the graphic elements without looking away from the road.

Returning to FIG. 1, the HUD device 102 of the HUD system 100 mayinclude a first projector 118, a second projector 120, a third projector122, and a fourth projector 124. The first projector 118 and the thirdprojector 122 may share a first beam splitter 126 and a first objectivelens 128, while the second projector 120 and fourth projector 124 mayshare a second beam splitter 130 and a second objective lens 132.Consequently, the output of the first projector 118 and the thirdprojector 122 may be received in the first beam splitter 126 andcombined into a singular output, which may be directed to (and through)the first objective lens 128. Similarly, the output of the secondprojector 120 and the fourth projector 124 may be received in the secondbeam splitter 130 and may be combined into a singular output, which maybe directed to (and through) the second objective lens 132.

The HUD device 102 may include an optional third beam splitter 134disposed downstream from the first and second objective lenses 128, 132and receive the output from the first and second objective lenses 128,132. The outputs from the first and second objective lenses 128, 132 maybe combined at the third beam splitter 134 into a singular output. Thesingular output may be a combination of the respective outputs of thefirst, second, third, and fourth projectors 118, 120, 122, 124. Further,the singular output may be directed to (and through) a third objectivelens 136 and an ocular lens 138 before being directed out of the HUDexit aperture 114 to the windshield 112 (both of FIG. 2). The windshield112 may be used as the display screen for the HUD system 100.

Each of the first projector 118, the second projector 120, the thirdprojector 122, and the fourth projector 124 may include a projector unit140, 142, 144, 146 and a diffuser screen 148, 150, 152, 154. Thediffuser screen 148, 150, 152, 154 may be rigidly fixed a set distancefrom the respective projector unit 140, 142, 144, 146 and arrangedrelative to the projector unit 140, 142, 144, 146 such that lightemitted from the projector unit 140, 142, 144, 146 passes through thediffuser screen 148, 150, 152, 154. The projector units 140, 142, 144,146 may be light-emitting units which project an image or graphicelement that passes through the associated diffuser screen 148, 150,152, 154. The diffuser screens 148, 150, 152, 154 may serve as aluminous image source (or object) for the other elements of the opticalsystem of the HUD device 102. Further, the diffuser screens 148, 150,152, 154 may help ensure that much of the light leaving the diffuserscreens 148, 150, 152, 154 falls into the optics following the diffuserscreens 148, 150, 152, 154 (e.g., the first beam splitter 126, the firstobjective lens 128, the second beam splitter 130, the second objectivelens 132, the third beam splitter 134, the third objective lens 136, andthe ocular lens 138), while spreading out light such that the lighteventually fills out the eye-box 116. Accordingly, brightness of theimage or graphic element(s) may remain constant while the driver's headmoves within the eye box 116. Accordingly, use of the diffuser screens148, 150, 152, 154 may prevent different parts of the image or graphicelement(s) from being visible from different points within the eye box116, and thereby may prevent the occurrence of different visual behaviorwith slight head movement.

The projector units 140, 142, 144, 146 may take the form of alight-emitting unit suitable for the herein-described use. The projectorunits 140, 142, 144, 146 may take the form of a light-emitting unitcapable of projecting an image or graphic element according to theherein-described use(s). Similarly, the diffuser screens 148, 150, 152,154 may take the form of a light diffusing screen suitable for theherein-described use(s).

The first projector 118 may be mounted on a first actuator 156 in theHUD device 102. The first actuator 156 may be a linear actuator capableof moving the first projector 118 in a linear direction toward and awayfrom the first beam splitter 126. Additionally, the third projector 122may be mounted on a second actuator 158 in the HUD device 102. Thesecond actuator 158 may be a linear actuator capable of moving the thirdprojector 122 in a linear direction toward and away from the first beamsplitter 126. The first and second actuators 156, 158 may take the formof linear actuators suitable for the herein-described use. The abilityof the first projector 118 and the third projector 122 to linearly movemay allow the first projector 118 and the third projector 122 to projectgraphic elements on dynamic or movable focal planes. In contrast to thefirst and third projectors 118, 122, the second and fourth projectors120, 124 may be fixedly arranged in the HUD device 102. The second andfourth projectors 120, 124 may project graphic elements on static focalplanes.

Using the first, second, third, and fourth projectors 118, 120, 122,124, the HUD device 102 may render graphic elements (contact-analogaugmented reality graphic elements or otherwise) in at least fourdistinct focal planes in the environment viewed by the driver throughthe windshield 112. The first projector 118 may project a first graphicelement 160 in a first focal plane 162 and the second projector 120 mayproject a second graphic 164 element in a second focal plane 166.Further, the third projector 122 may project a third graphic element 168in a third focal plane 170, and the fourth projector 124 may project afourth graphic element 172 in a fourth focal plane 174 (as will bedescribed with reference to FIGS. 3 and 4). The first, second, third,and fourth graphic elements 160, 164, 168, 172, and their associatedfirst, second, third, and fourth focal planes 162, 166, 170, 174 may berendered in the environment in view of the driver. The view of thedriver may be the view as the driver is driving the vehicle 106 and thedriver's eyes are in the eye box 116 while the driver is looking in aforward direction through the windshield 112.

Referring to FIG. 3 and FIG. 4, the projection of the first, second,third, and fourth graphic elements 160, 164, 168, 172 on the first,second, third, and fourth focal planes 162, 166, 170, 174 will bedescribed with reference to a ground surface 176 and a line-of-sight 178of the driver. The ground surface 176 is a surface of a road in front ofthe vehicle 106. For the purposes of this disclosure, the ground surface176 will be assumed to be a substantially planar surface. Theline-of-sight 178 of the driver is a line extending substantiallyparallel to the ground surface 176 from the eye box 116 in the forwarddirection. As used herein, a direction of the line-of-sight 178 is adirection extending toward and away from the driver and the vehicle 106along the line-of-sight 178.

The first focal plane 162 is a frontal focal plane, which may beoriented substantially perpendicularly to the line-of-sight 178 of thedriver. The third focal plane 170 is also a frontal focal plane, whichmay be oriented substantially perpendicularly to the line-of-sight 178of the driver. The first and third focal planes 162, 170 may be dynamicfocal planes which may be movable in the direction of the line-of-sight178, both in the forward direction (away from the vehicle 106) and in arearward direction (toward the vehicle 106). The second focal plane 166is a ground-parallel focal plane, which may be oriented substantiallyparallel to the ground surface 176, and may be disposed on the groundsurface 176 such that the second focal plane 166 is a ground focalplane. The fourth focal plane 174 is also a ground-parallel focal plane,which may be oriented substantially parallel to the ground surface 176,and is disposed above the ground surface 176. The fourth focal plane 174may be disposed above the ground surface 176 and the line-of-sight 178of the driver to be a sky or ceiling focal plane. As a result, thesecond and fourth focal planes 166, 174 may be static focal planes.

Referring to FIG. 4, the first, second, third, and fourth graphicelements 160, 164, 168, 172 may be used to present different informationto the driver. The exact type of information displayed by the first,second, third, and fourth graphic elements 160, 164, 168, 172 may vary.For exemplary purposes, the first graphic element 160 and third graphicelement 168 may present a warning to the driver instructing the driverto yield to a hazard or obstacle, or may present a navigationinstruction or driving instruction associated with rules of the road(e.g., a STOP sign, a YIELD sign, etc.). The second graphic element 164and fourth graphic element 172 may present navigation instructions tothe driver as a graphic overlay presented on the ground surface 176, ormay present a vehicle-surrounding indicator to the driver. The first,second, third, and fourth graphic elements 160, 164, 168, 172 maypresent information or graphic elements to the driver which aredifferent than those described herein, and a subset of the first,second, third, and fourth graphic elements 160, 164, 168, 172 may bepresented.

Returning to FIG. 1, the controller 104 may include one or morecomputers, (e.g., arithmetic) processors, or other devices capable ofcommunicating with one or more vehicle control systems 180 andcontrolling the HUD device 102. One or more of the vehicle controlsystems 180 (herein, “vehicle control system 180” or “vehicle controlcomponent 180”) may take the form(s) of the vehicle control system 180used to actively or passively facilitate control of the vehicle 106. Thevehicle control system 180 may include or communicate with one or moresensors (not shown) which may detect driving and environmentalconditions related to the operation of the vehicle 106.

With general reference to the operation of the HUD system 100, thecontroller 104 may communicate with the vehicle control system 180, andbased on the communication with the vehicle control system 180, maydetermine the type and position of graphic elements to be presented tothe driver of the vehicle 106. The controller 104 may determine the typeof graphic element to be presented as the first, second, third, andfourth graphic elements 160, 164, 168, 172 by the first, second, third,and fourth projectors 118, 120, 122, 124. Further, the controller 104may control the first, second, third, and fourth projectors 118, 120,122, 124 to project the first, second, third, and fourth graphicelements 160, 164, 168, 172 as the determined graphic elements. Thecontroller 104 may determine a target first graphic element position anda target third graphic element position as target positions at which thefirst and third graphic elements 160, 168 should be rendered in theenvironment to the driver. In addition, the controller 104 may controlthe first and second actuators 156, 158 to linearly move the first andthird projectors 118, 122 such that the first and third focal planes162, 170 may be moved to the target first and third graphic elementpositions, respectively.

Accordingly, the first projector 118 may project the first graphicelement 160 on the first focal plane 162. The first focal plane 162 maybe oriented substantially perpendicularly to the line-of-sight of thedriver, and may be movable toward and away from the vehicle 106 in thedirection of the line-of-sight 178 of the driver through linear movementof the first projector 118 by the first actuator 156. The secondprojector 120 may project the second graphic element 164 on the secondfocal plane 166, which may be static and oriented parallel to the groundsurface 176 and disposed on the ground surface 176. The third projector122 may project the third graphic element 168 on the third focal plane170. The third focal plane 170 may be oriented substantiallyperpendicularly to the line-of-sight of the driver, and may be movableor adjustable toward and away from the vehicle 106 in the direction ofthe line-of-sight 178 of the driver through linear movement of the thirdprojector 122 by the second actuator 158. The fourth projector 124 mayproject the fourth graphic element 172 on the fourth focal plane 174,which may be static, oriented parallel to the ground surface 176, andmay be disposed above the line-of-sight 178 of the driver. Thecontroller 104 may control the first and second actuators 156, 158 tomove the first and third projectors 118, 122 to move the first and thirdfocal planes 162, 170.

By having the first and third projectors 118, 122 project the first andthird graphic elements 160, 168 on the movable first and third focalplanes 162, 170 which are oriented substantially perpendicular to theline-of-sight 178 of the driver, focus of objects at different distancesfrom the vehicle 106 may be adjusted. This may facilitate the provisionof depth cues to the driver for the first and third graphic elements160, 168, especially since the HUD system 100 may be a vehicularapplication, with the vehicle 106 serving as a moving platform.

While the second and fourth projectors 120, 124 project the second andfourth graphic elements 164, 172 on the static second and fourth focalplanes 166, 174, the second and fourth focal planes 166, 174 may becontinuous. To make the second and fourth focal planes 166, 174 parallelto the ground surface 176, the diffuser screens 150, 154 of the secondand fourth projectors 120, 124 may be tilted. The optical system of theHUD device 102 may have very low distortion and may be nearlytelocentric for images in a ground-parallel focal plane. Therefore,light rays may be close to parallel with the optical axis, which allowsthe projected second and fourth graphic elements 164, 172 to beprojected or rendered without distorting or changing the magnificationwhile the second and fourth focal planes 166, 174 are tilted. Theresulting second and fourth graphic elements 164, 172 therefore appearon a continuous focal plane (the second and fourth focal planes 166,174) parallel to the ground surface 176. The second and fourth graphicelements 164, 172 may be rendered with an actual 3-dimensional (3-D)volumetric shape, instead of as line segments, to add monocular cues tostrengthen depth perception.

The continuous, static second and fourth focal planes 166, 174 mayfacilitate driver depth perception with regard to the second and fourthgraphic elements 164, 172. The continuous, static second and fourthfocal planes 166, 174 may allow for generation of real images or graphicelements through the forward-rearward direction in 3-D space (e.g., thedirection of the line-of-sight 178 of the driver), which also may allowproper motion parallax cues to be generated. Accordingly, as thedriver's head shifts from side-to-side or up-and-down, the second andfourth graphic elements 164, 172 may appear to the driver to be fixed inposition in the environment, rather than moving around. Consequently,the HUD system 100 might not need a head-tracking function to compensatefor movement of the driver's head.

With regard to the described exemplary information, which may bepresented to the driver, the vehicle control system 180 may includeprocessing and sensors capable of performing at least the followingfunctions: hazard or obstacle detection; navigation; navigationinstruction; and vehicle surrounding (e.g., blind-spot) monitoring. Thevehicle control system 180 may include processing and sensors capable ofperforming other vehicle control functions (e.g., highway merge assist,etc.), which may alternatively or additionally be tied to informationpresented to the driver using the HUD system 100. Regardless of thefunctions performed by the vehicle control system 180, the precisemanner of operation of the vehicle control system 180 to perform thefunctions, including the associated sensors and processing, may not berelevant to the operation of the HUD system 100.

The controller 104 may communicate with the vehicle control system 180and may receive therefrom inputs related to the operation of the vehicle106 and associated with the discussed functions and/or other functions.The controller 104 may control the HUD device 102 based on the inputsreceived from the vehicle control system 180. The controller 104 and/orthe vehicle control system 180 may determine the type of graphic elementto be displayed as the first, second, third, and fourth graphic elements160, 164, 168, 172. Further the controller 104 and/or the vehiclecontrol system 180 may determine the location of the first, second,third, and fourth graphic elements 160, 164, 168, 172; and which of thefirst, second, third, and fourth graphic elements 160, 164, 168, 172 areto be displayed. These determinations may be based on one or morevehicle functions employed by the driver, such as whether the driver isusing the navigation function, for example.

Regardless of which of the controller 104 or the vehicle control system180 are used to make these determinations, the controller 104 maycontrol the HUD device 102 to display the appropriate graphic elementsat the appropriate locations. This may include controlling the first,second, third, and fourth projectors 118, 120, 122, 124 to project theappropriate first, second, third, and fourth graphic elements 160, 164,168, 172. Additionally or alternatively, this may include controllingthe first and second actuators 156, 158 to linearly move the first andthird projectors 118, 122, to move the first and third focal planes 162,170 to the appropriate (e.g., target) positions. For example, one ormore actuators, such as 156, 158, may move one or more of the focalplanes, such as 162, 170. For example, with reference to the third focalplane 170, a distance between the third focal plane 170 and a windshieldof the vehicle 106 (e.g., at 302) may be adjusted by adjusting distance306. Similarly, distance 304 may be adjusted to change a target positionfor the focal plane 162.

Operation of the HUD system 100 will be described with reference to thevehicle 106 having the vehicle control system 180 which enables at leastthe following functions: a hazard or obstacle detection and warningfunction; a navigation function; a navigation instruction function; anda vehicle surrounding (e.g., blind-spot) monitoring function. Accordingto an embodiment, the vehicle 106 may have a subset of these functionsor additional functions, and the HUD system 100 may be employed withreference to the subset or additional functions. The description of theHUD system 100 with reference to these functions is merely exemplary,and used to facilitate description of the HUD system 100. Though thecontroller 104 and/or the vehicle control system 180 may makedeterminations associated with the operation of the HUD system 100, inthe below description, the controller 104 is described as makingdeterminations based on input received from the vehicle control system180.

Information related to the obstacle detection and warning function maybe presented to the driver as a contact-analog augmented reality graphicelement projected by the first projector 118 of the HUD device 102. Thevehicle control system 180 may detect various obstacles in the roadwayon which the vehicle 106 is travelling. For example, obstacles mayinclude pedestrians crossing the roadway, other vehicles, animals,debris in the roadway, potholes, etc. The detection of these obstaclesmay be made by processing information from the environment sensed bysensors (not shown) provided on the vehicle 106. Further, obstacledetection may be carried out in other manners.

When an obstacle is detected, the vehicle control system 180 maycommunicate obstacle information to the controller 104. The controller104 may receive the obstacle information from the vehicle control system180 and may determine the type of graphic element to present as thefirst graphic element 160 and the target first graphic element positionbased on the received obstacle information. While various types ofgraphic elements may be used, such as flashing icons, other signs, etc.,examples herein will be described with reference to a “YIELD” signpresented when an obstacle is detected.

Referring to FIG. 4, the obstacle detected by the vehicle control system180 may be a pedestrian 182 crossing the road on which the vehicle 106is travelling. In the exemplary view of the driver of FIG. 4, thevehicle 106 is traveling on a road, which is being crossed by thepedestrian 182. Accordingly, the vehicle control system 180 may sendobstacle information related to the pedestrian 182 to the controller104. Based on the obstacle information, the controller 104 may determinethe type of graphic element to be displayed as the first graphic element160; in this case, for example, the graphic element may be a “YIELD”sign, although other graphic elements may be used. The controller 104may determine the target first graphic element position such that thefirst graphic element 160 may perceived by the driver to be at a samedepth (e.g., focal plane) as the pedestrian 182. Further, the controller104 may adjust the target first graphic element position such that thefirst graphic element 160 ‘tracks’ or ‘follows’ the pedestrian 182, asthe pedestrian 182 walks, for example.

The controller 104 may control the first projector 118 to project the“YIELD” sign as the first graphic element 160. Further, the controller104 may control the first actuator 156 to linearly move the firstprojector 118 such that the first graphic element 160 may be perceivedby the driver (e.g., while the driver's eyes are in the eye box 116 andthe driver is looking in the forward direction through the windshield112) to be at the same depth as the pedestrian 182. The first actuator156 may be controlled such that the first graphic element 160 may beprojected on the first focal plane 162, which may be positioned at thetarget first graphic element position and may be oriented substantiallyperpendicular to the line-of-sight 178.

As the vehicle 106 and the pedestrian 182 travel on the road, therelative distance between the two may change. This change in distancemay be communicated to the controller 104 by the vehicle control system180. The target first graphic element position may be changedaccordingly and the first actuator 156 may be controlled by thecontroller 104 to move the first focal plane 162 to remain at the (e.g.,changed/changing) target first graphic element position. Accordingly,projecting the first graphic element 160 on the first focal plane 162(which may be movable in the direction of the line-of-sight 178 of thedriver) may allow the depth cues associated with the first graphicelement 160 to be reproduced so that the driver may accurately judge theposition of the first graphic element 160 (e.g., the detected obstacle).

Additionally, information related to the navigation function may bepresented to the driver as a contact-analog augmented reality graphicelement projected by the second projector 120 of the HUD device 102. Thevehicle control system 180 may, upon receiving a navigation request fromthe driver (e.g., the input of a desired location), generate anavigation route for the driver to follow to travel to the desiredlocation. The navigation route may include a set of driving directionsfor the driver to follow and may include instructions to turn ontostreets on the route to the desired location. The navigation functionmay be carried out in other manners than that discussed herein. When thenavigation function is activated, the vehicle control system 180 maycommunicate the driving directions associated with the navigationfunction to the controller 104.

The controller 104 may receive the driving directions from the vehiclecontrol system 180 and may determine the type of graphic element tooutput as the second graphic element 164. The types of graphic elementsassociated with the navigation function may include graphic elementswhich instruct the driver to continue on the current road (e.g., astraight line or arrow), to turn left or right onto an upcomingcross-road (e.g., a left/right arrow or line turning in the appropriatedirection), to enter, merge onto, or exit from a highway (e.g., a lineor arrow indicating the appropriate path), etc. The controller 104 mayselect the appropriate graphic element to output as the second graphicelement 164 based on the driving direction communicated from the vehiclecontrol system 180, for example.

Referring to the exemplary view of the driver of FIG. 4, the drivingdirection for the driving route determined by the navigation function ofthe vehicle control system 180 includes a left-hand turn onto anupcoming street. Accordingly, the controller 104 may control the secondprojector 120 to generate and project a left-hand turn graphic elementas the second graphic element 164 on the second focal plane 166. Asshown in FIG. 4, the second focal plane 166 may be oriented parallel tothe ground surface 176 and may be disposed on the ground surface 176.The second projector 120 may be fixedly arranged in the HUD device 102,such that the second focal plane 166 is static. Further, the secondfocal plane 166 may be continuous, such that the second graphic element164 may be rendered to the driver with appropriate depth cues as a 3-Dimage.

According to one or more embodiments, information related to thenavigation instruction function may be presented to the driver as acontact-analog augmented reality graphic element projected by the thirdprojector 122 of the HUD device 102. The vehicle control system 180 mayuse sensors or information stored in a database and associated with amap to monitor the road on which the vehicle 106 is traveling, and todetermine upcoming navigation instructions associated with travel onthat road. For example, the vehicle control system 180 may detect anupcoming required stop, yield, or other condition (herein, collectivelyreferenced as “road condition”) on the road on which the vehicle 106 istraveling. The vehicle control system 180 may determine a navigationinstruction associated with the detected road condition (e.g., a stopinstruction associated with a stop road condition, etc.). The navigationinstruction function may be carried out in various manners, thespecifics of which are not necessarily relevant to the operation of theHUD system 100. Additionally, road conditions may include, among otherthings, traffic on a road segment, obstructions, obstacles, weatherconditions, conditions of a surface of a road segment, speed limitsassociated with a portion of a road or road segment, etc. Roadconditions may generally include reasons to speed up, slow down, take adetour, stop, exercise caution, etc. while driving, for example.

The vehicle control system 180 may communicate the road condition or thenavigation instructions associated with the road condition, as well asinformation related to a position of the road condition, to thecontroller 104. The controller 104 may control the third projector 122to project the third graphic element 168 to communicate information tothe driver related to the road condition or associated navigationinstruction accordingly. The controller 104 may receive the roadcondition or navigation instruction information, as well as the positioninformation, from the vehicle control system 180, and may determine thetype of graphic element to present as the third graphic element 168 anda target third graphic element position.

Various types of graphic elements may be used in conjunction withnavigation instruction functions, for example: a STOP sign, a YIELDsign, a ONE WAY sign, a NO TURN ON RED sign, etc. The type of graphicelement may be selected to communicate the navigation instructionassociated with the road condition. Whichever type of graphic elementthe controller 104 determines should be used as the third graphicelement 168, that graphic element may be projected to appear at thelocation of the driving condition. The target third graphic elementposition may be determined as a position at which the third graphicelement 168 should be rendered in view of the driver based on theposition of the detected road condition relative to the vehicle 106.

The controller 104 may control the third projector 122 to project theappropriate graphic element as the third graphic element 168. Thecontroller may control the second actuator 158 to linearly move thethird projector 122 such that the third graphic element 168 is projectedand rendered to be perceived by the driver (e.g., while the driver'seyes are in the eye box 116 and the driver is looking in the forwarddirection through the windshield 112) to be at the same depth (e.g.,having a same focal plane) as the road condition. The second actuator158 may be controlled such that the third graphic element 168 isprojected on the third focal plane 170, which may be positioned at thetarget third graphic element position and may be oriented substantiallyperpendicularly to the line-of-sight 178. The controller 104 may controlthe second actuator 158 to continuously linearly move the thirdprojector 122. The third projector 122 may be moved so that the thirdfocal plane 170 moves as a distance between the vehicle 106 and thedetected road condition (e.g., the target third graphic elementposition) changes (as detected by the vehicle control system 180 andcommunicated to the controller 104), for example, as a result of thevehicle 106 driving toward the detected road condition.

In the exemplary view of from the perspective of the driver in FIG. 4,the vehicle 106 is approaching a four-way intersection at which thevehicle 106 should stop. Accordingly, the vehicle control system 180 maydetect the stop road condition at a position of an entrance of theintersection, and may determine the navigation instruction associatedwith the stop road condition to be a stop instruction. The stop roadcondition or instruction, as well as the position of the stop roadcondition, may be communicated to the controller 104, which maydetermine that a STOP sign should be presented as the third graphicelement 168. The controller 104 may determine that the third graphicelement 168 (e.g., the STOP sign) should appear at the position of theentrance of the four-way intersection. The position of the entrance ofthe intersection may therefore be determined to be the target thirdgraphic element position.

The controller 104 may control the third projector 122 to project the“STOP” sign as the third graphic element 168, and may control the secondactuator 158 to move the third projector 122 such that the third graphicelement 168 is projected and rendered to be perceived by the driver(e.g., while the driver's eyes are in the eye box 116 and the driver islooking in the forward direction through the windshield 112) to be atthe same depth as the entrance of the four-way intersection. The secondactuator 158 may be controlled such that the third graphic element 168may be projected on the third focal plane 170, which may be positionedat the target third graphic element position and oriented substantiallyperpendicularly to the line-of-sight 178. As the vehicle 106 travels onroad, the relative distance between the vehicle 106 and the entrance ofthe four-way intersection changes. This change in distance may becommunicated to the controller 104 by the vehicle control system 180,the target third graphic element position may be changed accordingly,and the second actuator 158 may be controlled by the controller 104 tomove the third focal plane 170 to remain at the (e.g., changed/changing)target third graphic element position. Accordingly, projecting the thirdgraphic element 168 on the third focal plane 170 which may be movable inthe direction of the line-of-sight 178 of the driver, the depth cuesassociated with the third graphic element 168 may thus be reproduced sothat the driver may judge the position of the third graphic element 168(e.g., the detected road condition).

Information related to the vehicle surrounding (e.g., blind-spot)monitoring function may be presented to the driver by the fourthprojector 124 of the HUD device 102. The vehicle control system 180 maydetect the existence of other vehicles in an area immediatelysurrounding or surrounding the vehicle 106. The detection of the othervehicles immediately surrounding the vehicle 106 may be made byprocessing information regarding the surroundings of the vehicle 106sensed by sensors (not shown) provided on the vehicle 106. The vehiclesurrounding determination may be carried out in various manners.

The vehicle surrounding information may be determined by the vehiclecontrol system 180 and may be communicated to the controller 104. Thecontroller 104 may receive the vehicle surrounding information from thevehicle control system 180 and may determine how, if at all, to modifythe fourth graphic element 172 projected on the fourth focal plane 174.The graphic element used as the fourth graphic element 172 to facilitatethe vehicle surrounding (e.g., blind-spot) monitoring function may be avehicle surrounding indicator, illustrated in FIG. 4.

The vehicle surrounding indicator may include a central marker that mayrepresent the vehicle 106 and (in this example) eight surroundingmarkers representing positions immediately surrounding the vehicle 106.The vehicle control system 180 may communicate information about thepositions of vehicles in the immediate surroundings of the vehicle 106,and the controller 104 may control the fourth projector 124 to changethe fourth graphic element 172 such that one or more of the associatedsurrounding markers are highlighted. The highlighting of the surroundingmarkers may indicate to the driver the position of other vehicles in theimmediate surroundings of the vehicle 106.

In FIG. 4, the fourth graphic element 172 may be projected on the fourthfocal plane 174, which may be oriented parallel to the ground surface176 and may be disposed above the ground surface 176 and theline-of-sight 178. The fourth projector 124 may be fixedly arranged inthe HUD device 102, such that the fourth focal plane 174 is static. Thefourth focal plane 174 may be continuous, such that the fourth graphicelement 172 may be rendered to the driver with depth cues as a 3-Dimage.

The fourth graphic element 172 may be presented in a form different thanthe vehicle surrounding indicator of FIG. 4. The fourth graphic element172 may be projected onto the fourth focal plane 174, which may beoriented parallel to the ground surface 176 and may be disposed abovethe ground surface 176 and the line-of-sight 178 of the driver.Accordingly, the fourth graphic element 172 may be provided on the skyfocal plane, which may be appropriate since the information communicatedby the fourth graphic element 172 need not interact with theenvironment.

The HUD system 100 may project graphic elements, some of which may beprojected as contact-analog augmented reality graphic elements. Thegraphic elements may be projected by the HUD system at continuouslychanging focal distances, as well as in ground-parallel focal planes,with continuous (or often) changing focus from front-to-back in thedirection of the line-of-sight 178 of the driver. Accordingly, depthperception cues may be provided in an attempt to facilitate focus andincrease attention to the environment. This may enable the driver (aswell as other vehicle occupants) to observe information presented viathe graphic elements at substantially the same time as observing theenvironment. In addition, spatial perception may be influenced by focalcues, and the focal plane adjusting capability, as well as thecapability to show graphic elements on continuous, staticground-parallel focal planes, of the herein-described HUD system 100 mayimprove spatial perception. Accordingly, an improvement in spatialperception may be observed when adjusting the focal cues as compared tothe spatial perception observed when a size of a graphic element isadjusted.

The configuration of the HUD device 102, including the use of the beamsplitters 126, 130, 134 and lenses 128, 132, 136, 138, may allow the HUDdevice 102 to have a relatively compact size. Further, the lenses 128,132, 136, 138 may allow a range of depth to expand from a few meters infront of the vehicle 106 to infinity within the physical space allocatedfor the optics of the HUD device 102. In addition, the beam splitters126, 130, 134 may be used as optical combiners to merge disparate setsof projected rays from the first, second, third, and fourth projectors118, 120, 122, 124 through the lenses 128, 132, 136, 138 to combineseparate images from the first, second, third, and fourth projectors118, 120, 122, 124 into one unified image (e.g., or graphic element)projected in view of the driver.

In one or more embodiments, the disclosed, as well as other features andfunctions, or alternatives or varieties thereof, may be combined intodifferent systems or applications. Additionally, various alternatives,modifications, variations, or improvements therein may be subsequentlymade by those skilled in the art.

For example, fewer or more projectors may be used in the HUD system 100to project fewer or more graphic elements. Further, while the HUD system100 is described as having two projectors which project graphic elementsin frontal focal planes and two projectors which project graphicelements in ground-parallel focal planes, the proportion of frontal andground-parallel focal planes may be changed. The described vehiclefunctions associated with the HUD system 100 are exemplary, and may bechanged or modified.

Further, the mechanism by which the frontal focal planes are moved maybe modified from that described herein. For example, rather than movingthe entire projector (e.g., the first and third projectors 118, 122using the first and second actuators 156, 158), merely the diffuserscreens (e.g., the diffuser screens 148, 152 of the first and thirdprojectors 118, 122) may be moved relative to the respective projectorunits (e.g., the projector units 140, 144).

Additionally, while the HUD system 100 has been described with referenceto the vehicle 106, which may be a four-wheeled automobile for outdooruse, the HUD system 100 may be used in different types of vehicles. Forexample, the HUD system may be provided in a marine vehicle (e.g., aboat), an air vehicle (e.g., an airplane or jet), or a vehicle intendedfor indoor use (e.g., a transportation cart, a vehicle used for materialhandling, such as a forklift, etc.).

FIG. 5 is an illustration of an example component diagram of a system500 for 3-D navigation according to one or more embodiments of thisdisclosure. The system 500 may include a HUD component 100, a vehiclecontrol component 180, a controller component 104, a navigationcomponent 540, a depth map component 550, a depth buffering component560, one or more sensor components 570, and one or more controller areanetworks (CANs) 580. The HUD component 100 may be a vehicular volumetricHUD system, such as the HUD system 100 of FIG. 1 and may includecomponents described herein. In one or more embodiments, the HUDcomponent 100 may be a 3-D HUD, a variable distance HUD, an augmentedreality HUD (AR-HUD), etc., as wells as other types of HUDs.

The navigation component 540 may receive or identify an origin location(e.g., point A) and one or more destination locations (e.g., point B).The navigation component 540 may calculate or determine one or moreroutes from point A to point B, for example. Generally, the navigationcomponent 540 is associated with a vehicle. For example, the navigationcomponent 540 may be mounted on the vehicle, integrated with one or moresystems or one or more components of the vehicle, housed within thevehicle, linked or communicatively coupled with one or more componentsof the vehicle, or located within the vehicle, etc. The navigationcomponent 540 may identify or receive the origin location and thedestination location. In one or more embodiments, the navigationcomponent 540 may include a telematics component (not shown) that maydetermine a current location or current position of the vehicle.

Additionally, the navigation component 540 may generate one or moreroutes from the origin location to one or more of the destinationlocations. In one or more embodiments, the navigation component 540 maygenerate one or more routes from a current location or current positionof the vehicle to one or more destination locations. A route of the oneor more routes may include one or more portions or one or more routeportions. As an example, one or more portions of the route may includeone or more navigation instructions or maneuvers associated with one ormore road segments or one or more intersections of road segments. One ormore portions of the route may include one or more turns, navigationmaneuvers, road segments, intersections, landmarks, or other elementsalong the route. The navigation component 540 may identify one or moreof these turns, navigation maneuvers, landmarks, etc. and issue one ormore navigation commands or one or more navigation instructionsaccordingly, such as to a driver of the vehicle.

The navigation component 540 may issue the one or more navigationcommands or navigation instructions via an audio prompt, visual prompt,tactile prompt, etc. For example, the navigation component 540 mayinterface with one or more peripheral components (not shown) bytransmitting one or more prompts across one or more controller areanetworks (CANs) 580. The navigation component 540 may play back anaudible instruction, such as, “Turn left at Main Street”, or flash alight on the left hand portion of a display, vibrate the steering wheel,etc. to indicate to a driver that a driving action should be taken. Thenavigation component 540 may interact with one or more other componentsto facilitate transmittal or delivery of one or more of the drivinginstructions.

For example, the HUD component 100 may project one or more navigationinstructions or one or more navigation maneuvers as one or more graphicelements or avatars in view of an occupant or driver of the vehicle.These navigation instructions may be received (e.g., directly orindirectly) from the navigation component 540. The HUD component 100 mayproject an avatar on successive focal planes such that the avatarappears to be moving to an occupant, such as a driver having a view fromeye box 116 of FIG. 2. The HUD component 100 may enable a driver toperceive a volumetric image in view of the driver, where the volumetricimage may serve as a ‘virtual’ guide vehicle for the driver of thevehicle to follow. It may appear to the driver of the vehicle that he orshe is merely following a guide vehicle to a destination location, forexample. Additionally, one or more other navigation commands ornavigation instructions may be projected as a volumetric placeholder,marker, or flagpole, as described herein.

The HUD component 100 may project one or more graphic elements, whichmay be contact analog augmented reality graphic elements, conformalaugmented reality graphic elements, avatars, icons, etc. These graphicelements may be projected by the HUD component 100 in a volumetricmanner. Thus, one or more visual cues or one or more depth cuesassociated with the graphic elements may be substantially preserved.Preservation of one or more of these visual cues or depth cues may beachieved by projecting or rendering graphic elements on a dynamic focalplane or a movable focal plane. The HUD component 100 may project orrender one or more graphic elements on a movable or adjustable focalplane. A dynamic focal plane or a movable focal plane may be moved oradjusted along a path or a line, such as a line of sight of an occupantof a vehicle, as discussed with reference to FIG. 1 and FIG. 3, forexample. The dynamic focal plane may be movable towards a vehicle or awindshield of a vehicle or away therefrom.

In one or more embodiments, a focal plane may be dynamic as a result ofmovement of projectors or screens of the HUD component 100, such asthrough the use of actuators, for example. One or more projectors of theHUD component 100 may move in a linear fashion, thereby enablingrespective projectors to project one or more graphic elements on adynamic, movable, or adjustable focal plane, which move when theprojectors move. In other embodiments, one or more other means oralternative means for adjustments may be utilized.

Explained another way, when a graphic element is projected on a dynamic,movable, or adjustable focal plane, the graphic element may be projectedonto a focal plane wherein a distance (e.g., distance 304 or distance306 of FIG. 3) from the focal plane and the vehicle is being adjusted.Projectors of the HUD component 100 may project or render graphicelements on movable focal planes, therefore, the focus of graphicelements projected at various distances from the vehicle may beadjusted. As mentioned, one or more of the focal planes may be orientedsubstantially perpendicular or substantially parallel to a line of sightof an occupant of the vehicle. A focal plane may be ground parallel orground perpendicular. Additionally, one or more of the focal planes maybe movable or static with respect to the line of sight of the occupantor the ground. This may enable depth cues associated with the graphicelements to be presented to occupants of the vehicle, such as thedriver, as the vehicle moves or travels (e.g., and thus serves as amoving platform), as discussed herein.

The HUD component 100 of FIG. 5 may project or render volumetriccontact-analog augmented reality graphic elements. Thus, these graphicelements may be projected to appear at various distances. The HUDcomponent 100 may project graphic elements at multiple focal planes orin an adjustable manner. Explained yet another way, focal planes ofgraphic elements projected by the HUD component 100 may be adjusted todistances that extend beyond the windshield, such as next to apedestrian on the sidewalk. This may enable an occupant of the vehicleto focus on the operating environment or driving environment, ratherthan switching focus of their eyes between the windshield or instrumentpanel of the vehicle and the driving environment.

Accordingly, graphic elements may be projected or visually placed (e.g.,by the HUD component 100) in an environment in direct view of anoccupant. Thus, graphic elements may be rendered in the same space asthe real environment, rather than on the windshield, allowing depth cuesassociated with the graphic element to be reproduced in a more accurateor correct manner. As a result, graphic elements may be projected on thesame focal planes as real world objects (e.g., the road) such that anoccupant of a vehicle may view the graphic elements without looking awayfrom the road, for example.

These multiple focal planes or adjustable focal planes may be achievedwhen projectors of a HUD component 100 are moved as light rays may bereshaped or altered such that a graphic element or virtual object beingprojected may appear to be further away than the windshield or have afocal plane that is not on the windshield. The projected graphic elementor virtual object may have similar focal properties as a real object(e.g., pedestrian, vehicle, sign, etc.) that is far away (e.g., tenmeters), for example. As light rays are reflected by glass of thewindshield, outgoing light rays diverge, thereby creating a ‘reflected’image or a real image, which may be projected as a graphic element.

Since the light rays may be reflected off of the windshield, rather thanbeing emitted or appearing from the windshield (e.g., as with specialcoatings), re-rendering of a graphic element is typically not necessarywhen an occupant moves his or her head. For example, the continuous,static focal planes of FIG. 3 may enable optically ‘correct’ or realimages to be generated through the forward-rearward direction in3-dimensional space (e.g., the direction of the line-of-sight of anoccupant), thereby allowing proper motion parallax cues to be generated.Accordingly, when the occupant's head shifts, graphic elementsassociated with these focal planes may appear to be fixed in position inthe environment, rather than moving around. As mentioned, the HUDcomponent 100 typically does not require head-tracking functionality tocompensate for movement of an occupant's head.

The HUD component 100 may be rastor based, rather than vector based.Thus, graphic elements projected by the HUD component 100 may be abitmap, may include a dot matrix structure, or may be a rectangular gridof pixels. Additionally, the HUD component 100 may project one or moreportions of one or more graphic elements with different shading,transparency levels, colors, brightness, etc.

Thus, the HUD component 100 may render or project graphic elements oravatars with various degrees of freedom. Accordingly, accommodation maybe preserved such that the eyes of an occupant may actively changeoptical power to focus on a graphic element projected on a focal plane.Similarly, vergence may be preserved such that the occupant may haveconcurrent inward rotation of a graphic element as the graphic elementis projected to move ‘closer’ (e.g., by projecting onto successivelycloser focal planes).

In one or more embodiments, the HUD component 100 may project a graphicelement, such as an avatar or a moving avatar, for a driver or occupantof a vehicle to follow as a navigation instruction, maneuver, orcommand. For example, the HUD component 100 may project or render one ormore of the graphic elements as a moving avatar, a placeholder, anidentifier, a flag pole, a marker, and so forth. These graphic elementsmay be projected on one or more focal planes around an environmentsurrounding the vehicle, and may be projected in view of an occupant ofthe vehicle. An avatar or graphic element projected by the HUD component100 may lead a driver of a vehicle through one or more portions of aroute, and attempt to mitigate collisions with obstacles, obstructions,or road conditions by being projected to weave, navigate, move, ortravel around the obstacles. A sensor component 570 may sense one ormore obstacles or road conditions and a controller component 104 maydirect the HUD component 100 to project the graphic element(s) such thatthe graphic element(s) travels around or bypasses a road condition, suchas by changing lanes to avoid a traffic barrel, for example.

In one or more embodiments, the sensor component 570 may sense,identify, or detect one or more road conditions in an environment aroundor surrounding the vehicle. The sensor component 570 may detect oridentify road segments, sidewalks, objects, pedestrians, other vehicles,obstructions, obstacles, debris, potholes, road surface conditions(e.g., ice, rain, sand, gravel, etc.), traffic conditions, traffic signs(e.g., red lights, speed limit signs, stop signs, railroad crossings,trains, etc.). An indication of these road conditions may be transmittedto the controller component 104 or the vehicle control component 180.For example, one or more of the CANs 580 may be used to facilitatecommunication between the sensor component 570 and the controllercomponent 104 or the vehicle control component 180. In one or moreembodiments, the sensor component 570 may include one or more imagecapture devices, a microphone, blind spot monitor, parking sensor,proximity sensor, presence sensor, infrared sensor, motion sensor, etc.

As used herein, a traffic condition may include an intersection, avehicle, such as a vehicle sharing a roadway with a vehicle equippedwith a 3-D navigation system, a railroad crossing, a red light, a roadsegment, a sidewalk, a stop sign, a yield sign, a traffic sign, a train,etc. As used herein, a road condition may include debris, gravel,potholes, a road surface condition, sand, fallen trees, spillage, oilslicks, weather conditions, such as ice or rain, etc. Further, an‘object’ may include various traffic conditions, road conditions,weather conditions, etc. Examples of objects may include but are notnecessarily limited to other vehicles, stores, buildings, landmarks,obstacles in a roadway, road segments, intersections, pedestrians, etc.Objects may be found, detected, or be associated with a path, one ormore road segments, etc. along a route on which a vehicle is travellingor is projected to travel along.

For example, the sensor component 570 may receive, sense, or detectinformation (e.g., objects, traffic conditions, road conditions, weatherconditions, etc.) from an environment, such as a driving environment, anoperating environment, or an environment surrounding a vehicle.Information collected by the sensor component 570 may be passed throughthe controller area network 580 and analyzed by the controller component104 or over a telematics channel provided by the navigation component540 and analyzed by a remote server (not shown). The informationreceived or collected by the sensor component 570 may be indicative of areason for a driver to exercise caution, a reason to slow down (e.g.,due to a speed limit posting or sign), a reason to speed up, a reason tostop, a reason to take a detour, etc. Explained another way, informationcollected by the sensor component 570 may be analyzed and translated bythe controller component 104 (or a remote server) to one or moresuggested driving actions for a driver. These suggested driving actionsmay be projected as visual cues or graphic elements by the HUD component100 based on information collected by the vehicle control component 180and determinations made by the controller component 104.

In one or more embodiments, the sensor component 570 may sense or detectone or more objects, one or more traffic conditions, one or more roadconditions, one or more weather conditions, etc. For example, aproximity sensor (e.g., one of the sensor components 570) on a vehiclemay detect a pedestrian crossing a crosswalk. Additionally, othercomponents may receive information relating to the environment, drivingenvironment, or operating environment. For example, the vehicle controlcomponent 180 may receive similar information (e.g., via a telematicschannel provided by the navigation component 540 or from the sensorcomponent 570 over the controller area network 580). The vehicle controlcomponent 180 may aggregate information from one or more sources. Thevehicle control component 180 may receive object information, trafficcondition information, road condition information, weather conditioninformation, etc.

The sensor component 570 may gather information directly from theenvironment, while the vehicle control component 180 may aggregateinformation from difference sources which may not necessarily be localto the vehicle. For example, the vehicle control component 180 mayreceive traffic information along a projected route for the vehicle froma traffic database or a news source. The vehicle control component 180may aggregate information about the environment, such as the environmentaround a vehicle or environment information associated with a projectedroute for the vehicle (e.g., environment information at a location wherea vehicle is anticipated to pass through while travelling along apredetermined route).

Further, the vehicle control component 180 may receive data associatedwith one or more of the road conditions or data related to anenvironment surrounding the vehicle (e.g., operating environment,driving environment, surrounding environment, etc.). In one or moreembodiments, the vehicle control component 180 may receive one or moreof the road conditions from the sensor component 570. Additionally, thevehicle control component 180 may receive one or more road conditionsfrom one or more other sources, such as a server (not shown) or adatabase (not shown), for example. The vehicle control component 180 maybe communicatively coupled with the server, third party, database, orother entity via a telematics channel initiated via a telematicscomponent (not shown). The vehicle control component 180 may gatherinformation associated with one or more portions of a route from anorigin location to a destination location.

For example, the vehicle control component 180 may receive roadcondition information that includes traffic information of a roadsegment (e.g., whether traffic is congested, if there is an accident onthe road, etc.). Additionally, the vehicle control component 180 mayreceive speed limit information associated with one or more of the roadsegments of a route. This information may be used to determine how toproject one or more graphic elements to a driver or occupant of avehicle. For example, if a road segment is associated with a 65 mphspeed limit, and a current velocity (e.g., detected by the sensorcomponent 570) of the vehicle is 25 mph, the vehicle control component180 may instruct the HUD component 100 to project an avatar such thatthe avatar appears to speed up upon turning onto the road segment.

As another example, if the sensor component 570 detects a traffic barrelin a current lane in which the vehicle is travelling, the vehiclecontrol component 180 may receive this information and make adetermination that a navigation instruction to change lanes should beprojected by the HUD component 100. This instruction may be transmittedover one or more CANs 580 to the HUD component 100, which may project,render, or animate an avatar or graphic element changing lanes orshifting position in response to the detected traffic barrel. Forexample, the HUD component 100 may project an avatar or icon thatappears to weave around or navigate around the traffic barrel, which ispositioned in front of the vehicle in the operating environmentsurrounding the vehicle. As well, the vehicle control component 180 mayinstruct the HUD component 100 project a turn signal on the avatar, as areal vehicle might indicate when changing lanes. Further, the vehiclecontrol component 180 may adjust a perceived velocity for the avatar asthe avatar approaches the traffic barrel. This may be achieved byprojecting the avatar or graphic element in successively closer focalplanes or by adjusting a dynamic focal plane of the graphic element suchthat the distance between the dynamic focal plane and the vehicle orwindshield of the vehicle is reduced. IN a similar manner, when it isdesired to render the avatar as speeding up, the dynamic focal plane maybe adjusted such that the distance between the dynamic focal plane andthe vehicle or windshield thereof is increased.

The vehicle control component 180 may receive one or more roadconditions. A road condition of the one or more road conditions mayinclude traffic information of one or more of the road segments or speedlimit information associated with one or more of the road segments.Further, the vehicle control component 180 may drive the HUD component100 in order to project one or more graphic elements based on one ormore of the road conditions, such as a speed limit of a road segment anda current velocity of the vehicle. Thus, the vehicle control system 180may determine one or more actions (e.g., stop, speed up, change lanes,slow down, etc.) or navigation instructions that should be projected bythe HUD component 100.

In one or more embodiments, the system 500 may include a view managementcomponent (not shown) that manages one or more aspects of one or moregraphic elements projected by the HUD component 100. In one or moreembodiments, the controller component 104 may manage one or more ofthese aspects or functionality associated with the vehicle controlcomponent 180. For example, the controller component 104 may receive oneor more road conditions.

The controller component 104 may determine a type of graphic element tobe displayed, projected, animated, rendered, etc. by the HUD component100. As an example, when a vehicle is travelling along one or moreportions of a route that includes relatively straight road segments, thecontroller component 104 may project a graphic element as an avatar. Theavatar may appear or be projected as a vehicle or a guide vehicle. In ascenario where a vehicle is travelling along one or more portions of aroute that includes one or more turns or other navigation maneuvers, thecontroller component 104 may command the HUD component 100 to project agraphic element to be a marker at a location associated with one or moreof the turns. For example, if a route includes a right turn from a firststreet onto a second street, the controller component 104 may commandthe HUD component 100 to project a marker or identifier at, to, around,etc. the intersection of the first street and the second street. Thecontroller component 104 may determine one or more types (e.g., markers,identifiers, flag poles, guide avatars, etc.) of graphic elements to bedisplayed.

Additionally, the controller component 104 may determine one or morelocations where a graphic element should be projected. For example, thecontroller component 104 may decide when and where a graphic elementshould be projected or how the graphic element should be displayed. Alocation of a graphic element may include a focal plane, a distance ofthe focal plane from the vehicle or windshield thereof, x-coordinates,y-coordinates, z-coordinates, etc. along an x, y, or z axis, forexample. This location may be referred to as a target position for oneor more of the graphic elements. In one or more embodiments, thecontroller component 104 may adjust a distance between one or more ofthe focal planes of one or more of the graphic elements and the vehicle(e.g., or windshield of the vehicle) based on one or more roadconditions associated with one or more portions of the route, a currentposition of the vehicle, a current velocity of the vehicle, etc.

For example, a road segment (e.g., portion of a route where a vehicle iscurrently located or positioned) is associated with a 65 mph speed limit(e.g., a road condition), and a current velocity (e.g., detected by thesensor component 570) of the vehicle is 25 mph (e.g., current velocityof the vehicle). Thus, the controller component 104 may command the HUDcomponent 100 to project an avatar or graphic element, which appears tobe travelling at about 65 mph. In one or more embodiments, the avatarmay be projected in a manner which demonstrates gradual accelerationfrom 25 mph to 65 mph. Thus, a distance between the focal plane of theavatar and the vehicle may be adjusted accordingly. For example, in ascenario where the vehicle accelerates at approximately the same pace,the distance between the focal plane and the vehicle may remain aboutthe same. If the vehicle accelerates at a slower pace than the avatar,the distance between the focal plane and the vehicle may be adjusted orincreased by the controller component 104. This adjustment may be basedon a current position of the vehicle or a current velocity of thevehicle, as well as road conditions of the route associated therewith.

Additionally, the controller component 104 may adjust or determine asize of a graphic element according to or based on a distance of thefocal plane of the graphic element and the vehicle with the HUDcomponent 100. The controller component 104 may adjust a height, size,width, depth, etc. of a graphic element, guide icon, or avatar based ona desired perception. For example, to make an avatar appear to speed up,the controller component 104 may adjust the size of the avatar to shrinkor be reduced while projecting the avatar onto successively fartherfocal planes or adjusting a dynamic focal plane to be farther andfarther away from the vehicle.

In one or more embodiments, the size of the graphic element may beutilized as an indicator for an importance level of a navigationinstruction or message. For example, the more important the message ornavigation instruction, the bigger the avatar, icon, or graphic elementmay be projected.

The controller component 104 may determine one or more actions for oneor more of the graphic elements to be projected by the HUD component100. For example, the controller component 104 may command the HUDcomponent 100 to project an avatar to speed up, slow down, stop, changelanes, activate a turn signal prior to changing lanes, flash, blink,change an orientation or angle of an avatar, change a color of anavatar, etc. Further, the controller component 104 may adjust targetpositions for one or more of the graphic elements based on roadconditions, a current position of the vehicle, a current velocity of thevehicle, or other attributes, characteristics, or measurements. In oneor more embodiments, the controller component 104 may interface orcommunicate with the navigation component 540 across one or more CANs580.

The controller component 104 may identify obstructions, distractions, orother aspects which may impede a driver or occupant of a vehicle. In oneor more embodiments, the controller component 104 may receive anindication of a location of the horizon, such as from the sensorcomponent 570, and project graphic elements above the horizon or skyplane, etc. The controller component may determine or adjust a color,transparency, or shading of one or more graphic elements based on a timeof day, traffic levels associated with the route, a familiarity thedriver has with the route, etc.

The depth map component 550 may build or receive a depth map of anenvironment around or surrounding the vehicle, such as an operatingenvironment. The HUD component 100 may utilize the depth map to projectone or more graphic elements accordingly. For example, an avatar isdisplayed are turning a corner and is ‘behind’ a building (e.g., abuilding is between the line of sight of an occupant of the vehicle anda perceived or target location of the graphic element or avatar).Therefore, the HUD component 100 may enable or disable projection of oneor more portions of the avatar or graphic elements in line with whatshould be seen if the avatar were a real-world object.

The depth map component 550 may receive a depth map from a server orthird party server. For example, the depth map component 550 maydownload a depth map from a server via a telematics channel initiatedvia a telematics component (not shown). In other embodiments, the sensorcomponent 570 may detect depth information, which may be used by thedepth map component 550 to build the depth map. The depth map component550 may interface or communicate with one or more sensors to build thedepth map or receive a pre-built depth map from a database. Thus, thedepth map component 550 may build or receive a depth map based on depthinformation. The depth map may be indicative of distances of one or moresurfaces, objects, obstructions, geometries, etc. in the environment orarea around the vehicle.

The depth map may be passed or transmitted to the controller component104, which may command the HUD component 100 to render one or more ofthe graphic elements accordingly. For example, the HUD component 100 mayproject or render graphic elements based on a height of an eye boxassociated with an occupant of a vehicle, a location of the vehicle, anda depth map of the area, which may be actively sensed or received from adatabase. The HUD component 100 may project one or more of the graphicelements based on the depth map to account for a perspective of one ormore occupants of the vehicle.

The depth buffering component 560 may facilitate perspective managementfor one or more occupants of the vehicle utilizing the depth mapgenerated or receive by the depth map component 550. Thus, the depthbuffering component may facilitate rendering of graphic elements suchthat the graphic elements appear visually ‘correct’ to an occupant. Forexample, if a graphic element is to be projected behind a real worldobject, the depth buffering component 560 may ‘hide’ a portion of thegraphic element from an occupant by not projecting or rendering thatportion of the graphic element. Thus, the depth buffering component 560may manage which portions (e.g., pixels) of a graphic element are drawn,projected, or rendered, and which portions are not. Accordingly, thedepth buffering component 560 may enable or disable rendering of one ormore portions of one or more of the graphic elements based on the depthmap.

Additionally, the depth buffering component 560 may obscure real worldobjects, thereby inhibiting what an occupant of a vehicle may see. Forexample, the depth buffering component 560 may command the HUD component100 to project a white graphic element such that the graphic elementoverlays a real world object, such as a billboard (e.g., detected bysensor component 570). As a result, an occupant may not see thebillboard or have an obscured view of the billboard. Thus, the depthbuffering component 560 may mitigate distractions for a driver or anoccupant of a vehicle by providing graphic elements that facilitatediminished reality.

Examples of navigation instructions that may be projected by the HUDcomponent 100 may include following a guide vehicle and speeding up(e.g., changing a dynamic focal plane to have an increased distance fromthe focal plane to the vehicle, thereby adjusting a near-far perceptiona driver or occupant may have of the graphic element). Other examplesmay include slowing down (e.g., adjusting the distance between a focalplane and the vehicle to be reduced), changing lanes (e.g., adjusting atarget position for a graphic element), navigating around obstructions,turning, arrival, marking a location, and so forth. As an example, thecontroller component 104 may command the HUD component 100 to project anavatar to ‘slow down’ if a pedestrian steps out onto the road segment,road way, crosswalk, etc. As another example, the controller component104 may command the HUD component 100 to project deceleration based onan angle of a turn, a speed limit associated with a road segment, roadconditions, such as ice, etc. For example, if there is ice on the roadsurface, the controller component 104 may command the HUD component 100to project an avatar moving slower than it would if no ice were presenton the road surface.

In one or more embodiments, the controller component 100 may mark oridentify an upcoming turn or intersection with a marker, flag post, flagpole, identifier, etc. For example, the HUD component 100 may render orproject a placeholder or marker according to the perspective of theoccupant of the vehicle. The depth map component 550 may provide a depthmap such that real life objects (e.g., buildings, trees, etc.) act asline of sight blockers for one or more portions of the placeholder. Asan example, if a placeholder has a perceived height of 100 feet, and a50 foot tall building is in front of the placeholder, the depthbuffering component 560 may compensate for the line of sight blocking bydisabling rendering or projection of a bottom portion of the placeholdergraphic element, thereby rendering the placeholder according to theperspective of the driver or occupant.

In one or more embodiments, one or more of the graphic elements may beprojected in view of an occupant of the vehicle based on the route(e.g., a follow a guide vehicle mode). In one or more embodiments, agraphic element may be projected as an avatar or other guide icon. Theavatar may appear to be flying and may be displayed against a real worldenvironment that is located around the vehicle. The avatar may move,travel, or ‘fly’ in 3-D space or in three dimensions. Thus, the avataror graphic element may appear to move in 3-D, thereby providing a moreintuitive feel or comfortable feeling for an occupant or driver that isfollowing the avatar. As an example, an avatar, graphic element, orguide icon may be projected such that it appears to change in height orsize based on a perceived distance from an occupant of the vehicle. Theavatar may be animated by sequentially projecting the moving avatar onone or more different focal planes. Additionally, the avatar mightappear to navigate around obstructions, obstacles, pedestrians, debris,potholes, etc. as a real vehicle would navigate. In one or moreembodiments, the avatar might ‘drive’, move, appear to move, etc.according to real-time traffic. The avatar may change lanes in a mannersuch that the avatar does not appear to ‘hit’ another vehicle orotherwise interfere with traffic. As another example, if a route takes adriver or a vehicle across train tracks, the avatar may stop at thetrain tracks when a train is crossing. In other embodiments, the HUDcomponent 100 may project the avatar or graphic element to stop at stopsigns, red lights, or obey traffic laws. Upon arrival at a destinationlocation, the HUD component 100 may render or project an avatar in aresting pose, for example.

Further, the system 500 for 3-D navigation may generate an intuitivemessage, instruction, or command for an occupant of a vehicle, such as adriver. The instruction may be based on one or more aspects related toperspective, as provided by the ability of the HUD component to projector render volumetric, 3-D graphic elements along one or more adjustablefocal planes. For example, the 3-D effect may be determined based ondistance, perspective, perceived distance, road conditions, etc.

FIG. 6 is an illustration of an example flow diagram of a method 600 for3-D navigation according to one or more embodiments of this disclosure.At 602, a route may be generated from an origin location to adestination location. In one or more embodiments, the origin location orthe destination location may be received via a telematics channel, suchas from a global positioning system (GPS) unit. At 604, one or moregraphic elements may be projected on one or more focal planes in view ofan occupant of a vehicle. The graphic elements may be displayed asavatars, images, icons, identifiers, markers, etc. Additionally, thegraphic elements may be based on one or more portions of the route. Forexample, the graphic elements may be projected at various distancesdepending on the portion of the route at which a vehicle may be located(e.g., a current position of the vehicle).

At 606, a distance between a focal plane and the vehicle may be adjustedbased on road conditions associated with one or more portions of theroute. Further, the distance may also be adjusted based on a currentvelocity of the vehicle. For example, if a vehicle is traveling along aportion of a route associated with a 65 mile per hour (mph) speed limitand the current velocity of the vehicle is 25 mph, the distance betweenthe focal plane of a projected graphic element or avatar may beincreased (e.g., to indicate to the driver or occupant to speed up). Forexample, the graphic element may be projected to appear as if it weretravelling about 65 mph, thereby prompting the occupant or driver tospeed up and ‘catch’ the avatar (e.g., simulating the action offollowing a guide vehicle).

FIG. 7A is an illustration of an example avatar 700 for 3-D navigationaccording to one or more embodiments of this disclosure. The avatar 700of FIG. 7A may appear in front of a vehicle and appear to fly, glide,move, maneuver, etc. around elements, obstructions, traffic, roadconditions, etc. FIG. 7B is an illustration of an example avatar(s) 710for 3-D navigation according to one or more embodiments of thisdisclosure. The avatar(s) 710 of FIG. 7B are shown from an elevatedview, such as a birds-eye view slightly behind the avatars(s) 710. Asillustrated, one or more of the avatars 710 may be projected on one ormore different focal planes or target positions, thereby providing theperception that a driver or occupant is following a real vehicle.

FIG. 8A is an illustration of an example avatar 800 for 3-D navigationaccording to one or more embodiments of this disclosure. The avatar 800of FIG. 8A is illustrated as rotated counterclockwise to indicate a leftturn. FIG. 8B is an illustration of an example avatar 810 for 3-Dnavigation according to one or more embodiments of this disclosure. Inan example, the avatar 810 of FIG. 8B may indicate a left turn byblinking, flashing, changing color, etc. For example, the left wing ofthe paper airplane avatar 810 may glow or change in intensity toindicate the upcoming left turn. In one or more embodiments, an avatarmay be projected on focal planes closer to the vehicle such that itappears that the avatar is ‘slowing down’ prior to making a turn.

FIG. 9A is an illustration of an example avatar 900 for 3-D navigationaccording to one or more embodiments of this disclosure. FIG. 9B is anillustration of an example avatar 910 for 3-D navigation according toone or more embodiments of this disclosure. The avatar 900 of FIG. 9Amay be projected as a navigation instruction for a driver of a vehicleto slow down, for example. In FIG. 9B, the avatar 910 is illustrated asbeing projected above the horizon or a sky plane. Thus, the avatar 910does not obstruct the driver or occupant from viewing one or moreportions of the environment around the vehicle.

FIG. 10 through FIG. 20 are now described with reference to one or moreof the previous figures, such as the system 500 for 3-D navigation ofFIG. 5 or the side view 300 or focal planes of FIG. 3.

FIG. 10A is an illustration of an example scenario 1000 associated with3-D navigation according to one or more embodiments of this disclosure.In FIG. 10A, an occupant, driver, passenger, operator, etc. of a vehiclemay provide a command to one or more components of the vehicle, such asthe sensor component 570 of FIG. 5. In one or more embodiments, thesensor component 570 may include one or more peripheral components, oneor more peripherals, one or more interfaces, one or more interfacecomponents, such as a touch screen, a keyboard, one or more buttons, aninterface, a microphone, one or more image capture devices, gesturerecognition device, etc. For example, a vehicle may be equipped with atouch screen interface or buttons on a dashboard may be utilized tointerface with a menu. One or more components or the sensor component570 may capture or receive commands given or provided by an occupant ofa vehicle.

For example, a driver may communicate navigation commands to thevehicle, such as “How do I get to downtown Sunnyvale?” In this example,the sensor component 570 may include a microphone, which receives theverbal or spoken request from the occupant or driver and passes therequest to a controller area network 580 to a controller component 104or a navigation component 540. In other embodiments, if a mobile deviceis communicatively coupled with a vehicle, an occupant may utilize amicrophone of the mobile device, a touch screen of the mobile device, akeyboard or keypad of the mobile device, etc. to interface with thesystem 500 for 3-D navigation. The controller component 104 may performvoice recognition on the request by utilizing an on-board speechrecognition module to convert the request from speech to text (STT).According to some embodiments, the navigation component 540 may utilizea telematics channel to communicate with a remote server where STTprocessing may be performed.

In one or more embodiments, the HUD component (such as the HUD componentor system 100 of FIG. 1 or the HUD component 100 of FIG. 5) may project,present, render, or display confirmation of a request or a command. Forexample, the HUD component 100 may operate as a display component or maysupplement other display components for a vehicle. Continuing with the“How do I get to downtown Sunnyvale?” example in FIG. 10A, when acommand or a request is received by the controller component 104, thecontroller component 104 may determine whether or not to projectconfirmation of the request based on environment information provided ordetected by the sensor component 570. In this example, the sensorcomponent 570 does not detect moving vehicles, pedestrians, or otherobstacles, therefore, the controller component 104 may make adetermination that a confirmation of the request should be projected bythe HUD component 100. Additionally, the controller component 104 maydetermine a size, a shape, a model, a transparency, a color scheme, aheight, a width, a depth, a target position, a focal plane, etc. for acorresponding graphic element. A confirmation of the request may beprojected by the HUD component 100 in a text box, as illustrated at1004.

In one or more embodiments, the controller 104 may command the HUDcomponent 100 to project a graphic element as an avatar 1002, which mayappear as a guide icon. In FIG. 10A, the graphic element may beprojected as an avatar 1002 having a shape or appearance of an airplane,which glides along a route slightly ahead of a driver or occupant of thevehicle. To facilitate awareness, the controller component 104 mayinstruct the HUD component 100 to animate the avatar 1002 such that theavatar 1002 hovers or is constantly in motion. For example, to make theavatar 1002 easier to see, the controller component 104 may instruct theHUD component 100 associate the avatar 1002 with motion on a periodicbasis or other basis. For example, the HUD component 100 may project theavatar 1002 to appear as if the avatar 1002 is floating in water,bobbing, hovering, etc. Thus, driver awareness may be increased.

FIG. 10B is an illustration of an example scenario 1010 associated with3-D navigation according to one or more embodiments of this disclosure.When a request or a command, such as a navigation command, is receivedby the controller component 104, the controller component 104 mayforward the request or command to the appropriate component or module.In this example, the controller component 104 may identify the requestor command as a navigation command and pass the navigation command alongto the navigation component 540. Accordingly, the navigation component540 may determine an origin location for the vehicle (e.g., a currentlocation, a future start location), the destination location, andcalculate a route from the origin location to the destination location.As discussed herein, the route may include one or more route portionsalong one or more road segments and/or one or more navigation actions ormaneuvers.

The navigation component 540 may provide the route, route portions,and/or route information to the controller component 104, which maydetermine whether or not and/or how to render the route information. Forexample, in FIG. 10B, the vehicle is not in motion (e.g., in park) or istravelling at a low velocity (e.g., below a velocity threshold detectedor provided across the controller area network 580). Therefore, thecontroller component 104 may determine that a high-level view 1030 ofthe route may be appropriate for the HUD component 100 to project oroutput. Accordingly, the controller component 104 may instruct the HUDcomponent project the high-level view 103, which occupies a larger areawithin the environment. For example, the controller may determine a sizefor a graphic element or whether or not to project the graphic elementbased on a velocity of the vehicle. Conversely, if the vehicle is inmotion or travelling above a threshold velocity, the controllercomponent 104 may make a determination that a smaller version beprojected or that the HUD component 100 not project the high-level view1030 at that time.

As illustrated in FIG. 10B, the HUD component 100 may project ahigh-level view 1030 of a map, which includes a route 1032 from anorigin location to a destination location along one or more roadsegments. The HUD component 100 may project a compass 1034, road segmentidentifiers 1036, traffic information, estimated arrival times,estimated travel times, a current location of the vehicle, etc. In oneor more embodiments, the HUD component 100 may project different aspectsof the high-level view 1030 utilizing one or more color schemes. Forexample, the HUD component 100 may project the route such that the route1032 is rendered utilizing a first color and other road segments, suchas the road segment identified at 1036, are rendered utilizing a secondcolor.

In one or more embodiments, the sensor component 570 may include imagecapture devices or other sensors, which aid in determination of a colorpalette or color scheme for display or projection of graphical elements.For example, in FIG. 10B, there are one or more trees in the backgroundor in the environment, which may make green more difficult to see orvisualize. Accordingly, the controller component 104 may select a colorscheme for projection of one or more graphical elements. The colorscheme may utilize colors which contrast green, such as yellow, forexample. As another example, during the daytime, the sky may be sensedto be blue by the sensor component 570, while at night the sky mayappear black. The sensor component 570 may receive, capture, sense, ordetect color information associated with the environment, which may beutilized for contrast of graphic elements. The controller component 104may utilize this information to determine colors for which the HUDcomponent 100 may project one or more graphic elements. For example, thecontroller component 104 may determine that red is an option during thedaytime while blue is an option at nighttime. The controller componentmay determine color schemes or a color for a graphic element or avatarbased on a time of day or color information associated with theenvironment (e.g., received or detected by the sensor component 570).

In one or more embodiments, the HUD component 100 may animate one ormore aspects of the high-level view 1030. For example, the HUD component100 may animate the route 1032 along one or more road segments as if theroute 1032 was being drawn from the origin location to the destinationlocation with one continuous pen stroke. For example, the HUD component100 may render animation that represents travel along the route in asnake-like fashion from the origin location to the destination location.

The HUD component 100 may project graphic elements or avatars, such asthe avatar 1002 of FIG. 10A using different camera angles, cameraperspectives, or different views. Further, the HUD component 100 mayshift or transition between these views in an animated or gradualfashion. For example, the HUD component 100 may shift the cameraperspective or the camera angle from a birds-eye-view, such as thehigh-level view 1030 of FIG. 10B to a first-person view or third personview, as illustrated in FIG. 10A. For example, the HUD component 100 maygradually adjust the camera perspective to zoom in or fly in from abirds-eye-view, layout view, overhead view, zoomed-out view, orthird-person view to a zoomed-in view, first-person view, street-levelview, etc.

For example, the HUD component 100 may utilize a camera perspective or acamera angle which appears to finish zooming or transitioning at astreet-level view which coincides with the real world-view. This realworld-view may be the same view of the environment which an occupant ordriver would have sitting in his or her vehicle looking out through thewindshield into the real world. For example, the street-level view maybe a view an occupant has while within an eye box and whilefacing/looking in a forward direction through a windshield of thevehicle. The zoom animation or transition may result in a rendering ofan avatar or graphic element in view of the driver, such as the avatar1002 of FIG. 10A. FIG. 11A is an illustration of an example scenario1100 associated with 3-D navigation according to one or more embodimentsof this disclosure. The view of the avatar 1002 provided in this examplescenario 1100 is a transitional view that is between the birds-eye-viewor high-level view 1030 of FIG. 10B and the third-person view in FIG.10A.

In one or more embodiments, the HUD component 100 may provide a zoomanimation in reverse order (e.g., from third-person view tobirds-eye-view) when an occupant requests a view of a map or when theoccupant initiates a map view command. In this scenario, the HUDcomponent 100 may project a third-person view of an avatar (e.g., 1002of FIG. 10A) and zoom out or fly away to a bird-eye-view (e.g.,high-level view 1030 of FIG. 10B). Further, the HUD component 100 mayadjust an orientation of the route, map, or view based on a direction orbearing of the vehicle. For example, the HUD component 100 may orientthe map or view such that the direction of travel along a current roadsegment appears to be up or forward, for example.

FIG. 11B is an illustration of an example scenario 1110 associated with3-D navigation according to one or more embodiments of this disclosure.The HUD component 100 may provide navigation instructions and/orsuggested navigation maneuvers to a driver of a vehicle. For example,the navigation component 540 may generate a list of navigationinstructions for a route from an origin location to a destinationlocation. The controller component 104 may instruct the HUD component100 to output respective navigation instructions in the environment,such as at 1130, for example. Navigation instructions may includedirections for a driver, such as turn right, turn left, merge right,merge left, yield, stop, slow down, speed up, accelerate, etc. Further,navigation instructions may not necessarily direct a driver to takeaction. For example, navigation instructions may alert a driver of avehicle that no turns on red are permitted.

In one or more embodiments, the controller component 104 may utilize acolor scheme or instruct the HUD component 100 project one or more ofthe graphic elements or avatars utilizing different colors to representone or more navigation actions. For example, when the HUD component 100projects an avatar, such as avatar 1002A, 1002B, or 1002C of FIG. 11B asgreen, this may indicate that no objects, obstacles, or oncoming trafficare detected by the sensor component 570. As another example, the HUDcomponent 100 may project one or more of the avatars 1002A, 1002B, or1002C as red to indicate to the driver that it is not yet safe to makethe turn. Thus, the HUD component 100 may project a “yield” as anavigation instruction rather than “turn right,” for example.

As illustrated in FIG. 11B, the HUD component 100 may project multipleavatars 1002A, 1002B, and 1002C. In this example, the avatar 1002A maybe projected on a first focal plane, the avatar 1002B may be projectedon a second focal plane, and the avatar 1002C may be projected on athird focal plane. The first focal plane is closest to the vehicle,followed by the second focal plane, then the third focal plane, therebygiving the appearance that avatar 1002C is the farthest away from thevehicle. Since the HUD component 100 is capable of projecting multipleavatars 1002A, 1002B, and 1002C on different focal planes, this enablesthe system 500 for 3-D navigation to provide a projected route, path, orpredetermined route for a driver to follow.

In one or more embodiments, the HUD component 100 may project one ormore of the avatars 1002A, 1002B, or 1002C such that respective avatarsmaintain focal planes which are static relative to the current positionof the vehicle. The HUD component 100 may project avatars or graphicelements such that the distance between the vehicle and focal planes forrespective avatars decreases as the vehicle ‘approaches’ an avatar. Forexample, the HUD component 100 may project an avatar, such as one of1002A, 1002B, or 1002C as a stationary object relative to the vehicle,thereby making it appear to a driver or occupant that an avatar ahead ofthe vehicle at one time is being passed or waits for the vehicle to‘catch-up’ at a later time. In these embodiments, the HUD component 100may project additional avatars as a route progresses and an individualor driver of a vehicle ‘passes’ the stationary or static avatars.

In one or more other embodiments, the HUD component 100 may project oneor more of the avatars 1002A, 1002B, or 1002C such that respectiveavatars have focal planes which are dynamically adjusted as to thecurrent position of the vehicle or have a constant or fixed distancefrom the vehicle. For example, the HUD component may project avatars orgraphic elements such that the distance between the vehicle and focalplanes for respective avatars remains constant as the vehicle moves. TheHUD component may project avatars as stationary objects, which appear tomove along with the vehicle. In one or more embodiments, the HUDcomponent 100 may transition between projecting avatars as stationaryobjects relative to the vehicle and projecting avatars as objects whichappear to move with the vehicle.

The HUD component 100 may project any one of a plurality of graphicelements or avatars as an animation by adjusting focal planes on whichto project one or more of the graphic elements or avatars. As anexample, positioning of avatars 1002A, 1002B, and 1002C may be achievedby expanding a single avatar, such as the avatar 1002 of FIG. 11A orFIG. 10A. For example, one or more avatars (or graphic elements) may beexpanded from a first avatar and respective avatars may be collapsedinto a single avatar. The HUD component 100 may cause multiple avatarsto appear to fly-out, separate, advance, or be projected ahead of afirst avatar to facilitate animation of route navigation and/ornavigation instructions, for example.

In one or more embodiments, the HUD component 100 may project one ormore graphic elements as pointers, which may alert a driver or anoccupant of a vehicle of one or more objects, obstacles, roadconditions, etc. The controller component 104 may select a pointer typebased on a velocity of the vehicle and/or a velocity of an object. Forexample, if the vehicle is stationary, oncoming traffic (e.g., or otherobjects) may be identified by projecting a graphic element which rotatesin a manner which points to the traffic, hazard, or moving object whilethe object approaches the vehicle or departs away from the vehicle.

FIG. 12A is an illustration of an example scenario 1200 associated with3-D navigation according to one or more embodiments of this disclosure.The sensor component 570 may detect passing traffic or another vehicle1214. In this example, since the trajectory or path of the other vehicle1214 is on a collision course with the suggested navigation actionindicated by avatar 1002B, the controller component 104 may instruct theHUD component 100 to project navigation instructions that alert thedriver that he or she should yield at 1130. For example, the sensorcomponent 570 may track the location of the other vehicle 1214. Thisinformation may be transmitted to the controller component 104, whichmay determine that pointer graphic element should be projected by theHUD component 100 at 1002A. The HUD component 100 may thus project(e.g., output) the pointer graphic element 1002A such that it tracks orfollows 1212A the other vehicle 1214.

FIG. 12B is an illustration of an example scenario 1210 associated with3-D navigation according to one or more embodiments of this disclosure.Continuing with the example from FIG. 12A, as illustrated, the othervehicle 1214 has passed the vehicle, which was stationary. The HUDcomponent 100 may continue to project the yield navigation instruction1130 and the next navigation instruction or action (e.g., turn right) at1002B. In this case, the sensor 570 component may detect the othervehicle 1214 and the HUD component 100 may project the avatar 1002B in acorresponding color, such as red. The controller component 104 mayselect a color for a graphic element to be projected by the HUDcomponent 100 based on object information and/or environment information(e.g., collected or detected by the sensor component 570). The pointer1002A of FIG. 12B may continue pointing or tracking the other vehicle1214 by rotating along a z-axis in a manner corresponding to a trackedobject. Thus, the controller component 104 may adjust a yaw of thepointer graphic element 1002A.

FIG. 13A is an illustration of an example scenario 1300 associated with3-D navigation according to one or more embodiments of this disclosure.While an individual or driver is driving a vehicle, the navigationcomponent 540 may determine a current location of a vehicle and/orcorresponding coordinates. Utilizing this information, the navigationcomponent 540 or controller component 104 may determine a name of a roadand instruct the HUD component 100 to project the road name on a focalplane corresponding to a surface of the road or road segment, asillustrated at 1330. As discussed herein, projection of the graphicelement on the road surface may be performed in a manner that appearsstationary with respect to the vehicle or in a manner that appears tomove with the vehicle. In one or more embodiments, the HUD component 100may project the road name 1330 in response to a query from an occupantof the vehicle, such as “What road am I on?”, for example. Additionally,other graphic elements, information, or highlighting may be projected onlandmarks, objects, etc.

In FIG. 13A, multiple graphic elements or avatars are projected by theHUD component 100 and may be animated, collapsed, expanded, etc. asdiscussed herein. Thus, avatars 1002A, 1002B, and 1002C may be providedto guide a driver of a vehicle or other occupants along a route to adestination. FIG. 13B is an illustration of an example scenario 1310associated with 3-D navigation according to one or more embodiments ofthis disclosure. As illustrated, avatars 1002A and 1002B are projectedwith varying pitch angles, roll angles, or yaw angles. The controllercomponent 104 may select or adjust a pitch, a roll, or a yaw angle foran avatar or a graphic element based on various considerations. Suchconsiderations include, a trajectory of a path, a radius of an upcomingturn, a current velocity of a vehicle, a speed limit associated with anupcoming turn, suggested navigation action, information from theenvironment, distance until next navigation instruction or action, etc.

In this example, a proposed route has the vehicle taking an upcomingexit ramp with a circular road pattern or road segment, therefore, thecontroller component 104 may instruct the HUD component to projectavatar 1002B with a greater roll angle or yaw angle than avatar 1002A,thereby providing a perception that there is an upcoming right turn.Additionally, the controller component 104 may select a shorter distancebetween the focal planes for avatars 1002A and 1002B to indicate atightness of a turn. In other embodiments, the controller component 104may select a color (such as yellow or orange) for one or more of theavatars to indicate an upcoming turn and/or a corresponding reason for adriver to slow down, for example.

FIG. 14A is an illustration of an example scenario 1400 associated with3-D navigation according to one or more embodiments of this disclosure.A driver or occupant may provide the system 500 for 3-D navigation witha request or a command while engaged in navigation from an originlocation to a destination location. The navigation component 540 maysupport addition of one or more waypoints between the origin locationand the destination location such that navigation to the destinationlocation is not interrupted in a manner which requires re-setting (orreconfiguring) the navigation to the destination location. For example,when a waypoint is inserted, the navigation component 540 may calculatea revised route from a current location of the vehicle to the waypointand from the waypoint to the destination location (which was previouslydetermined).

In FIG. 14A, navigation from an origin location to a destinationlocation is ongoing, as indicated by the avatar 1002, which is directingthe driver of the vehicle to continue straight through the currentintersection. At 1404, confirmation of the request or command may beprojected in a text box. In one or more embodiments, the controllercomponent 104 may not instruct the HUD component 100 project the textbox 1404. For example, the text box 1404 may not be provided whenmultiple obstacles are detected by sensor component 570, when thevehicle is travelling over a threshold velocity, or based on userpreferences, for example.

In one or more embodiments, the system 500 may interface with one ormore additional components, servers, other systems, such as a mobiledevice or remote servers to determine one or more of the waypoints. Forexample, a driver may request the system 500 to “take me to the nearestgas station”, “the cheapest gas station nearby,” or “a gas station on myway home”. According to these requests, the controller component 104 mayinterface with the navigation component 540 to determine a currentlocation for the vehicle, a destination location, a route between thecurrent location and the destination location, a gas price application,such as an application on a mobile device, etc. to determine pricing,location, or other criteria.

FIG. 14B is an illustration of an example scenario 1410 associated with3-D navigation according to one or more embodiments of this disclosure.The controller component 104 may determine a location of a waypoint orselect an appropriate waypoint from one or more available waypoints inconjunction with the navigation component 540. The navigation component540 may calculate or reroute the vehicle to the waypoint as a currentdestination location and set the previous destination location as thenext destination location. Further, the navigation component 540 maycalculate additional travel time associated with the waypoint. The HUDcomponent 100 may project this information (e.g., detour which will addapproximately eight minutes extra drive time) in text box 1430A, forexample.

As discussed herein, the HUD component 100 may provide or projectnavigation instructions 1130 and at least one avatar 1002 on one or morefocal planes in the environment. Further, the navigation component 540may utilize a telematics channel to receive or download metadata orinformation associated with a waypoint or destination. For example, thenavigation component 540 may receive a logo associated with thewaypoint, such as a logo for a store or a gas station, etc. The HUDcomponent 100 may project or output this logo or other relatedinformation at 1430B. Further, the HUD component 100 may animate one ormore aspects, such as by folding the logo 1430B into a plane shape orotherwise transforming the logo 1430B into the avatar 1002 or othergraphical element. The HUD component 100 may animate, transform,transition, etc. a graphical element into another graphical element.

FIG. 15 is an illustration of an example scenario 1500 associated with3-D navigation according to one or more embodiments of this disclosure.The HUD component 100 may project navigation instructions in a text box1130 and one or more avatars 1002A, 1002B, 1002C, 1002D, 1002E, etc.Additionally, the HUD component 100 may emphasize a route or pathprovided by respective avatars 1002A, 1002B, 1002C, 1002D, or 1002E bysequentially flashing the avatars, for example. The HUD component 100may project one or more of the avatars 1002A, 1002B, 1002C, 1002D, or1002E such that the avatars appear to navigate, drive, or move aroundobjects detected by the sensor component 570. The object may includehazards, road conditions, other vehicles, and so on. The logo 1430B ofFIG. 14B may be animated to appear to shrink to an icon sized logo “L”,as illustrated in FIG. 15. For example, the controller component 104 maydetermine a time period for the logo 1430B to be displayed at a firstsize or at full size. After expiration of the time period, the HUDcomponent 100 may project the logo “L” at a smaller size, icon size, ora second size. The controller component 104 may determine the timeperiod for the logo based on the distance to the corresponding object(e.g., restaurant, store, landmark, etc.), for example.

In one or more embodiments, early warning detection may be provided forone or more obstacles or objects not yet in view or range of an occupantof a vehicle when the occupant is looking out of the front of thevehicle through the windshield. FIG. 16A is an illustration of anexample scenario 1600 associated with 3-D navigation according to one ormore embodiments of this disclosure. In this example, traffic conditioninformation may be received by the vehicle control component 180 for oneor more traffic conditions along a route on which a vehicle istravelling, or is anticipated to travel. The vehicle control component180 may receive traffic condition information indicative of constructionalong one or more route portions of a route ahead of the vehicle. Forexample, construction may be occurring along a road segment on which thevehicle will travel if navigation instructions are followed. Thus,occurrence of this construction may be received by the vehicle controlcomponent 180 prior to the vehicle arriving at the construction.

The HUD component 100 may project navigation instructions 1130, whichmay include text, arrows, blinking arrows, and so forth. Additionally oralternatively, avatars 1002A and 1002B may be projected. In one or moreembodiments, the controller component 104 may determine a stopping pointor line where it is suggested to stop the vehicle. For example, thecontroller 104 may determine the stopping point based on a position ofanother vehicle, obstacles in the environment, other environmentinformation, etc. The controller component 104 may instruct the HUDcomponent 100 project this information as a horizontal line 1630 topromote safe driving, for example. In other embodiments, the HUDcomponent 100 may project a vertical line or other graphical element,which may be indicative of a safe following distance between thedriver's vehicle and another vehicle ahead, for example.

Accordingly, the controller component 104 may identify this obstacle(e.g., object) or traffic condition and cause the HUD component 100 toproject an advance notification or early warning graphic element 1602.This enables the system 500 to alert a driver or occupant of a vehicleof one or more objects, hazards, obstacles, traffic conditions, etc.even when respective objects are not directly visible when viewed orlooking out of the windshield (or other window) of the vehicle.

FIG. 16B is an illustration of an example scenario 1610 associated with3-D navigation according to one or more embodiments of this disclosure.As illustrated in FIG. 16B, the traffic condition or obstructionassociated with 1602 is ahead in the distance, and may not be apparentto a driver of a vehicle. Accordingly, the HUD component 100 mayidentify the hazard, construction, or object (e.g., received by thevehicle control component 180) by projecting an advance notificationgraphic element 1602 which points to and identifies the constructionrelative to the vehicle. For example, in FIG. 16A the construction is onthe left side of the vehicle, thus, the pointer or tail associated withthe advance notification graphic element 1602 may point outside of thewindshield viewing area. This may alter a driver or other occupant ofthe vehicle that construction is ahead.

The controller component 104 may determine one or more target positionsand update or adjust a target position for one or more of the graphicelements projected by the HUD component 100. For example, the controllercomponent 104 may select or determine a target position for the advancenotice graphic element 1602 based on a distance of the vehicle from theobject associated with or pointed at by the advance notice graphicelement 1602. Thus, the controller component 104 may determine that theconstruction (e.g., object) associated with the advance notice graphicelement 1602 is greater than a threshold distance away from the vehicle.For example, the navigation component 540 may be utilized to determine acurrent GPS position for the vehicle and compare the position withcoordinates for the construction provided by the vehicle controlcomponent 180. If the construction is determined to be greater than thethreshold distance, the HUD component 100 may project the advance noticegraphic element 1602 in the sky. For example, the HUD component 100 mayutilize a pointer having a length greater than a threshold length,rather than projecting a graphic element proximate, overlapping, oradjacent to the construction.

FIG. 17A is an illustration of an example scenario 1700 associated with3-D navigation according to one or more embodiments of this disclosure.As a vehicle approaches an object (e.g., the construction indicated byadvance notice graphic element 1602), the controller component 104 mayadjust the positioning or target position for the respective graphicelement (e.g., 1602). Additionally, the controller component 104 maychange the type of graphic element displayed. For example, asillustrated in FIG. 17A, the advance notice graphic element 1602 mightnot include a pointer, as previously utilized in FIG. 16A or FIG. 16B.For example, if the proximity between the vehicle and the object (e.g.,construction) is less than a threshold distance, the controllercomponent 104 may determine that the pointer may be omitted. Thus, theHUD component 100 may project the avatar 1002, the advance noticegraphic element 1602, and/or a hazard graphic element 1704A.

For example, when the vehicle control component 180 or the sensorcomponent 540 identifies an object which is a hazard, obstruction,obstacle, or is otherwise undriveable (e.g., a road segment underconstruction, etc.) the controller component 104 may instruct the HUDcomponent 100 project a hazard graphic element. For example, the hazardgraphic element 1704A may be projected or output in a manner thatoverlays the hazard. The depth buffering component 560 may facilitateinhibiting one or more portions of the hazard in the real world orenvironment such that the hazard is obscured from view of an occupant,operator, or driver of the vehicle. For example, the HUD component 100may project graphic elements that are opaque or have varying degrees oftransparency. When the controller component 104 determines an area,hazard, or object may not (or should not) be driven across or on, theHUD component 100 may project hazard graphic elements accordingly to“fill in” such areas.

FIG. 17B is an illustration of an example scenario 1710 associated with3-D navigation according to one or more embodiments of this disclosure.In FIG. 17B, the vehicle has approached the hazard or constructionillustrated in FIG. 17A, and the HUD component 100 has adjusted theperspective of the hazard graphic element 1704B accordingly. Forexample, the vehicle is closer in distance to the construction,therefore, the hazard graphic element 1704B of FIG. 17B is larger inarea than the hazard graphic element 1704A of FIG. 17A. Informationreceived from the sensor component 540 or the vehicle control component180 related to or associated with the hazard, construction, or objectsin the environment may be utilized by the controller component 104 todetermine a various parameters. These parameters may include size,shape, model (e.g., guide icon, avatar, obscure, transparency, color),height, width, depth, focal plane, color, and/or target position for agraphic element, or may identify overlap between graphic elements ordepth map information. These attributes may be based on objectinformation or environment information received by the vehicle controlcomponent 180 or the sensor component 570. In one or more embodiments,the controller component 104 may select or change colors of graphicelements based on oncoming traffic, time of day, colors of objects inthe environment, traffic rules, etc.

The depth map component 550 may manage or build a depth map of theenvironment or objects in the real-world environment around a vehicle.The controller component 104 may utilize the depth map to determinewhether an object should be visible, transparent, opaque, etc. withregard to a graphic element. For example, the construction (hidden underhazard graphic element 1704A or 1704B) may include traffic barrels,traffic cones, debris, potholes, digging, undriveable road surfaces,etc. In this example, the controller component 104 has made adetermination that these aspects are to be hidden under the hazardgraphic elements 1704A and 1704B. However, other objects in theenvironment may not necessarily be hidden under or layered under thehazard graphic elements 1704A and 1704B. For example, if a pedestriancrosses through the construction, the controller component 104 maydetermine that the pedestrian is an object that should not be obscured.Thus, the HUD component 100 may project the hazard graphic element 1704Bsuch that the hazard graphic element 1704B does not obscure thepedestrian from view of the driver while the pedestrian is walking‘through’ the hazard graphic element 1704B. Thus, the controllercomponent 104 may determine visibility, layering, transparency level,opaqueness level, etc. of a graphic element based on movement of anobject within an environment or other data received from the sensorcomponent 540 or the vehicle control component 180, such as identity ofan object, object type, object size, etc., for example. The depth mapinformation may be utilized by the controller component 104 or the HUDcomponent 100 to provide an occupant, operator, or driver of a vehicle arealistic perception of one or more graphic elements with respect to thesurrounding environment or the real world.

FIG. 18A is an illustration of an example scenario 1800 associated with3-D navigation according to one or more embodiments of this disclosure.When one or more objects are detected (e.g., by the sensor component570) to be in a path of a vehicle or along an anticipated route (e.g.,by environment information received by the vehicle control component180), the controller component 104 may instruct the HUD component 100highlight or track respective objects with graphic elements and/orpointers. For example, as a vehicle approaches a right turn (indicatedby navigation instructions 1130 and avatars 1002A, 1002B, and 1002C),the sensor component 570 may detect a pedestrian 1832 near a roadsegment on which the vehicle is anticipated to turn. Therefore, thecontroller component 104 may direct the HUD component 100 to project agraphic element 1834 on the pedestrian 1832. In one or more embodiments,the HUD component 100 may project a graphic element over the pedestrian1832. Further, the HUD component 100 may project a graphic element 1804,which illustrates signage indicative of a pedestrian road crossing.

As the vehicle and the pedestrian 1832 travel along the road segment,the relative distance between the two may change (e.g., the pedestrian1832 may move, the vehicle may move, or both the pedestrian 1832 and thevehicle may move, etc.). The sensor component 570 may detect this changein distance and communicate the change or update the distance with thecontroller component 104 or the vehicle control component 180. Thecontroller component 104 may update, change, or track target positionsfor one or more of the graphic elements (e.g., 1832 or 1834). Thus, thecontroller component 104 may adjust one or more focal planes for one ormore of the graphic elements based on object information or environmentinformation. As discussed herein, an actuator associated with projectinga graphic element on a focal plane may be controlled by the controllercomponent 104 to move, adjust, or change the focal plane (e.g., movingthe focal plane along a direction of line-of sight of a driver oroccupant, such as the line-of-sight 178 of FIG. 3).

FIG. 18B is an illustration of an example scenario 1810 associated with3-D navigation according to one or more embodiments of this disclosure.The HUD component 100 may project multiple graphic elements 1834 and1844 for multiple pedestrians 1832 and 1842, respectively. Additionally,the HUD component 100 may project one or more graphic elements asindicators, such as at 1802 or 1804.

FIG. 19A is an illustration of an example scenario 1900 associated with3-D navigation according to one or more embodiments of this disclosure.In one or more embodiments, the system for 3-D navigation 500 may enablean occupant of a vehicle to interact with objects in the environment. Anoccupant may ask questions (“what is that”, “That looks neat”) or makecomments (e.g., verbally or via text input) and receive graphicalelement projections as answers. For example, the sensor component 570may accept one or more queries, requests, or commands from a user,occupant, or driver of a vehicle. In this example, an occupant of thevehicle has asked, “What is that on the right?” The HUD component 100may display confirmation or repeat the query or request using a text boxgraphic element 1904. In one or more embodiments, queries or requestsmay be enabled during navigation, as illustrated by avatar 1002. One ormore other components may be employed to facilitate fulfillment of oneor more of the requests or commands. For example, if a mobile device iscommunicatively coupled with the system 500, the controller component104 may cause the mobile device to dial a telephone number of a businesswhen ordered to “call the restaurant on the right.” As another example,the controller 104 may interact with a restaurant reservationapplication installed on the mobile device or on the vehicle in responseto “make me a reservation at restaurant X.” Other commands or requestsmay utilize audio systems of the vehicle, such as “read me the menu.”

FIG. 19B is an illustration of an example scenario 1910 associated with3-D navigation according to one or more embodiments of this disclosure.The controller component 104 may process the request or route therequest associated with 1904 to the navigation component 540, todetermine objects within proximity of the vehicle, such as businesses,restaurants, etc. The controller component 104 may receive informationassociated with an object, such as a restaurant from a map application,a remote server, etc. This information may include metadata associatedwith the object, such as hours, a logo, a status, reviews for thebusiness, contact information, a telephone number, a menu, etc. The HUDcomponent 100 may project a logo graphic element 1430B, information 1930(e.g., telephone number, status, hours, etc.), and a highlightinggraphic element 1932, which may draw attention to the location of theobject (e.g., restaurant or business).

FIG. 20A is an illustration of an example scenario 2000 associated with3-D navigation according to one or more embodiments of this disclosure.In one or more embodiments, the navigation component 540 may provide thevehicle control component 180 with metadata associated with one or moreobjects in the environment. For example, one or more of the objects inthe environment may be buildings or businesses. The navigation component540 may download or receive address information for respective buildingsor businesses, which may be projected as graphic elements by the HUDcomponent 100. For example, addresses for buildings 2002, 2004, and 2006may be presented by the HUD component 100. In one or more embodiments,the controller component 104 may enable address graphical elementpresentation when a vehicle is within a threshold distance of adestination location or upon user request, for example. In otherembodiments, address graphical elements may be projected during anavigation mode, such as when directions are being provided by thenavigation component 540, as indicated by the avatar 1002.

FIG. 20B is an illustration of an example scenario 2010 associated with3-D navigation according to one or more embodiments of this disclosure.In this example, the sensor component 570 or the navigation component540 may identify one or more parking spots. As a result, the controllercomponent 104 may instruct the HUD component 100 project a graphicelement 2030 indicative of parking in view of an occupant of thevehicle. This graphic element 2030 may appear as an advance notificationgraphic element, include a pointer, have one or more transparencyattributes, color, shape, size, etc., or include object metadata (e.g.,cost of parking, time limit for spot, etc.) as described herein.

FIG. 21 is an illustration of an example flow diagram of a method 2100for 3-D navigation according to one or more embodiments of thisdisclosure. The method starts, at 2102. At 2104, one or more objects maybe tracked or metadata associated with respective objects may bereceived. Additionally, information related to an environment or routefrom an origin location to a destination location may be received. At2106, one or more focal planes or updated focal planes may be calculatedfor graphic elements to be projected in association with one or more ofthe objects. At 2108, one or more graphic elements may be rendered orprojected at one or more of the focal planes or on respective objects.At 2110, a determination may be made as to whether or not to continuetracking or projecting graphic elements in association with one or moreof the objects. If it is determined that additional tracking is desired,the method 2100 returns to 2104. If no additional tracking is desired,the method may end at 2112.

FIG. 22 is an illustration of an example, non-limiting system 2200configured for multi-level driver monitoring and/or control according toone or more embodiments of this disclosure. The system 2200 may includeat least one memory 2202 that may store computer executable componentsand/or computer executable instructions. The system 2200 may alsoinclude at least one processor 2204, communicatively coupled to the atleast one memory 2202. The at least one processor 2204 may facilitateexecution of the computer executable components and/or the computerexecutable instructions stored in the memory 2202. The term “coupled” orvariants thereof may include various communications including, but notlimited to, direct communications, indirect communications, wiredcommunications, and/or wireless communications.

It is noted that although the one or more computer executable componentsand/or computer executable instructions may be illustrated and describedherein as components and/or instructions separate from the memory 2202(e.g., operatively connected to the memory 2202), the various aspectsare not limited to this implementation. Instead, in accordance withvarious implementations, the one or more computer executable componentsand/or the one or more computer executable instructions may be stored in(or integrated within) the memory 2202. Further, while variouscomponents and/or instructions have been illustrated as separatecomponents and/or as separate instructions, in some implementations,multiple components and/or multiple instructions may be implemented as asingle component or as a single instruction. Further, a single componentand/or a single instruction may be implemented as multiple componentsand/or as multiple instructions without departing from the exampleembodiments.

The system 2200 may include the HUD component 100 that may rendervolumetric contact-analog augmented reality graphic elements withaccurately reproduced depth cues, as discussed in this detaileddescription. For example, the HUD component 100 may provide3-dimensional or “3-D” graphic elements that are rendered to appear tobe in the same space as the real-world environment.

The controller component 104 may be communicatively connected to the HUDcomponent 100 in some aspects, or may be integrated within the HUDcomponent 100 in other aspects. The controller component 104 maycommunicate with one or more vehicle control systems 180, as discussedin this detailed description.

The navigation component 540, which may be a navigation manager, maygenerate one or more routes between a first location and at least asecond location, as discussed in this detailed description. For example,the first location may be a starting point and the second location maybe a destination. According to some implementations, more than onedestination may be desired and, therefore, the navigation component 540may calculate multiple routes and/or calculate a single route withvarious waypoints or stops along the route.

The navigation component 540 may also provide one or more navigationcommands and/or one or more navigation instructions associated withvarious portions of the route(s), as discussed herein. The navigationcommands and/or navigation instructions may be rendered by thenavigation component 540 (or another system 2200 component) through anaudio prompt, a visual prompt, a tactile prompt, or in anotherperceivable format. In an example, the navigation component 540 maytransmit one or more prompts through one or more controller areanetworks (CANs) 580.

The system 2200 may also include the depth map component 550, which maybe a map displayer, that may output a map that includes the route. Themap displayer (or depth map component 550) may also output a map that isindicative of distances of one or more surfaces, objects, obstructions,or geometries in an area around the vehicle. The depth map component 550may output the map on a windshield of the vehicle. For example, the mapmay be transparent such that the map does not obstruct a view of realworld objects in the vehicle environment.

According to an implementation, the depth map component 550 provides adepth map of the vehicle environment, as discussed in this detaileddescription. The depth map may be utilized to output (e.g., by the HUDcomponent 100) one or more graphic elements within the rendered vehicleenvironment. According to some implementations, the depth map component550 may output the map on a windshield (e.g., overlaid on thewindshield), wherein the map is transparent and does not obstruct a viewof the real world objects through the windshield.

The depth buffering component 560 may coordinate perspective managementfor the vehicle occupants such that the graphic elements appear visually“correct” for each occupant, as discussed herein.

The one or more sensor components 570 may detect obstacles within the(real-world) vehicle environment. Further, the one or more sensorcomponents 570 may detect road conditions. The obstacles and the roadconditions may be utilized to provide suggested actions for the driverto perform. The suggested actions may be output in any perceivableformat including, for example, visual cues, graphical elements, verbalinstructions, and so on.

According to some implementations, at least one sensor component may bea user interface 2206. The user interface 2206 may allow one or morevehicle occupants to input various information, modify information, andperform other actions. Further, the user interface 2206 may output orrender information to the vehicle occupants in a perceivable format(e.g., visual, audible, tactile, and so on).

For example, FIG. 23A illustrates an example scenario 2300 related tointerfacing with a vehicle occupant according to one or more embodimentsof this disclosure. From a perspective of vehicle occupants, real-worldobjects may be perceived through a windshield 2302 (or through aheads-up display). As illustrated, a portion of the vehicle dashboardand/or hood, nearby vehicles, trees, a road or parking lot surface, andso forth may be perceived.

Displayed on the windshield 2302 may an indication of a geographiccoordinate 2304, which may be overlaid on the real-world items. In thisexample, the geographic coordinate 2304 is indicating that the vehicleis heading north. Further, although illustrated at a middle, top portionof the windshield 2302, the geographic coordinate 2304 may be located atother portions of the windshield.

Also displayed on the windshield 2302 may be an indication of roads thatare within a certain distance of the current vehicle location. The roadindication may be overlaid on the real-world items. One road of thedisplayed roads (e.g., map) is labeled at 2304. Further, labelsassociated with the roads may also be displayed. Such labels includeroute names, road numbers, or other manners of identification.

As illustrated, a first location 2308 (e.g., a start location) and asecond location 2310 (e.g., an end location) may be identified. Forexample, at a strategic level, high level commands (e.g., received by avehicle occupant) may be utilized to change a destination, to change aroute preference (e.g., scenery, time), to avoid certain situations(e.g., highways, intersections, identified roads), and so on.

According to an implementation, the vehicle occupant may communicatewith the system 2200 (of FIG. 22) component, such as through the userinterface 2206, to select a destination and choose a route. Thedestination and/or route may be chosen on a visual display 2312 of FIG.23B, which illustrates another example scenario 2314 related tointerfacing with a vehicle occupant according to one or more embodimentsof this disclosure. As illustrated, a vehicle occupant may perceive thedisplay 2312. For example, the display 2312 may be output through theuser interface 2206 (of FIG. 22). In another example, the display 2312may be a screen located on a dashboard, cluster, or another locationwithin the vehicle. In another example, the display 2312 may bepresented through the use of a heads-up display.

As illustrated, the display 2312 may output at least a portion of aroute 2316. As shown, the route 2316 may indicate the path traveled, aswell as a future path expected to be traveled as viewed from aperspective of a current location 2318. The route may also illustrate atarget location 2320 with a direction indicator 2322 that providesinformation as to a travel direction toward the target location 2320.The illustrated direction indicator 2322 is an arrow, however otherdirection indicators may be utilized. For example, the directionindictor 2322 may be words (e.g., go east, go straight, and so on),symbols (e.g., arrows, starts, or other shapes), color changes (e.g.,green to indicate a future travel route and blue to indicate the routealready traveled), and so on.

As illustrated, other information may be provided, such as an estimatedtravel time 2324 to reach the target location 2320. Also provided may bean address 2326 associated with the target location 2320. A nextdirection related to the route 2328 (e.g., the next turn, such as N101exist Ellis Street) to be traveled may also be provided. Further, otherinformation may be provided, such as a current time 2330, a currenttemperature 2332, or other information (e.g., traffic flow, weather, andso on).

The additional information provided (e.g., estimated travel time 2324,traffic flow, weather, and so on) may assist the vehicle occupant todecide routing constraints. If there is a situation that the vehicleoccupant wants to avoid (e.g., get to the destination faster than theestimated travel time 2324), the vehicle occupant may selectively changethe route.

The selection of the destination location and/or changes to the routemay be performed through interaction with the user interface 2206.According to some implementations, the user interface 2206 (as well asother interface components discussed herein) may provide a graphicaluser interface (GUI), a command line interface, a speech interface,Natural Language text interface, and the like. For example, a GUI may berendered that provides a user with a region or means to load, import,select, read, and so forth, various requests and may include a region topresent the results of the various requests. These regions may includeknown text and/or graphic regions that include dialogue boxes, staticcontrols, drop-down-menus, list boxes, pop-up menus, as edit controls,combo boxes, radio buttons, check boxes, push buttons, graphic boxes,and so on. In addition, utilities to facilitate the informationconveyance, such as vertical and/or horizontal scroll bars fornavigation and toolbar buttons to determine whether a region will beviewable, may be employed. Thus, it might be inferred that the user didwant the action performed.

The user may also interact with the regions to select and provideinformation through various devices such as a mouse, a roller ball, akeypad, a keyboard, a pen, gestures captured with a camera, a touchscreen, and/or voice activation, for example. According to an aspect, amechanism, such as a push button or the enter key on the keyboard, maybe employed subsequent to entering the information in order to initiateinformation conveyance. However, it is to be appreciated that thedisclosed aspects are not so limited. For example, merely highlighting acheck box may initiate information conveyance. In another example, acommand line interface may be employed. For example, the command lineinterface may prompt the user for information by providing a textmessage, producing an audio tone, or the like. The user may then providesuitable information, such as alphanumeric input corresponding to anoption provided in the interface prompt or an answer to a question posedin the prompt. It is to be appreciated that the command line interfacemay be employed in connection with a GUI and/or API. In addition, thecommand line interface may be employed in connection with hardware(e.g., video cards) and/or displays (e.g., black and white, and EGA)with limited graphic support, and/or low bandwidth communicationchannels.

FIG. 24A illustrates an example scenario 2400 related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure. As the vehicleis driven down a road, one or more avatars may be rendered and appear tobe in front of the vehicle. Although the avatars are illustrated asappearing above the vehicle, the avatars may be at another location(e.g., near street level, at about the same level as the vehicle, in thesky, on one or both sides of the vehicle, or any other location).

As illustrated, a first avatar 2402 may direct the operator of thevehicle toward the route. It is noted that although a certain number ofavatars are illustrated and described, the various aspects are notlimited to this amount of avatars and fewer or more avatars may beutilized during implementation. Further, the color, shape, or otherperceivable aspects of the avatars are not limited to what has beenshown and described herein.

Also provided may be an optional driving instruction 2404, such as“merge right.” In this example, a second avatar 2406 instructs thevehicle operator to proceed straight ahead. Further, a third avatar 2408instructs the operator to merge right.

FIG. 24B illustrates another example scenario 2410 related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure. This examplescenario 2410 is a continuation of the example scenario 2400 of FIG.24A. As illustrated a fourth avatar 2412 directs the vehicle operator tocontinue forward. A fifth avatar 2414 and a sixth avatar 2416 provideinstructions to the vehicle operator to merge to the right. For example,the fifth avatar 2414 and the sixth avatar 2416 may be positioned tovisually appear to be located over a turn (or exit lane) that isslightly out of view of the vehicle occupants.

FIG. 24C illustrates another example scenario 2416 related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure. An optionaltext description 2418 may be rendered to provide instructions to thevehicle operator. In this example, the optional text description 2418instructs the vehicle operator to “turn left.” A seventh avatar 2420points in a forward direction while an eighth avatar 2422 points towardthe left, the direction the vehicle is to turn.

As illustrated in FIG. 24A, FIG. 24B, and FIG. 24C for immediate view,planned vehicle maneuvers may be rendered. These planned vehiclemaneuvers may be rendered on a HUD (e.g., 2-D, 3-D) or another type ofrendering device. A vehicle may be directed to stay in a particularlane, change lanes, make turns, pull over, and so on. Vehicles maneuversmay be at a medium-level description of what to do, but not at anoperational level. If a driver of an autonomous vehicle, for example,does not agree with a decision provided by the system 2200, the drivermay take action to modify the autonomous system 2200 decision. Forexample, the driver may intervene and override decisions by inputtinginformation (e.g., through the user interface 2206). In a specificexample, the driver may verbally state “stay in this lane” or “turnright instead.”

According to some implementations, the vehicle occupant may usegestures, alone or at substantially the same time as other form or input(e.g., speech, touch screen, and so on). For example, FIG. 24Dillustrates a further example scenario 2424 related to allowing avehicle occupant to monitor and/or control navigation informationaccording to one or more embodiments of this disclosure. In thisexample, another avatar 2426 is provided as well as an optional textdescription 2828, which in this example indicates that there are “quickturns ahead.” As illustrated, the driver 2430 (or another occupant) maygesture the desired action. For example, as illustrated at 2432, thedriver 2430 is pointing in the desired direction of travel). However,other gestures may be utilized, such as a thumbs-down signal to indicateno, nodding the head to indicate yes, and so on).

For example, one or more sensor components 570 and/or the user interface2206 (both of FIG. 22) may take advantage of the motion and spatialexpression of gestures to indicate spatial relationship information.Thus, one or more sensor components 570 (or another system component,such as the user interface 2206) may detect the gesture or otherinformation (e.g., verbal commands, changes input directly onto a touchscreen, and so on). This information may be conveyed to a modificationcontroller 2208, which may determine whether the change requested by thedriver or vehicle operator may be performed.

According to an implementation, the modification controller 2208 mayalter at least a portion of the route based on an indication of a changereceived at the user interface 2206 and/or a sensor component 570. Inaddition, the navigation component 540 may regenerate the route betweenthe first location and the second location as a function of theindication of the change. In an implementation, the indication of thechange is received from the driver and/or from a passenger in thevehicle.

The indication of the change may be based on a gesture detected by theuser interface 2206 and/or the sensor component 570. According toanother implementation, the indication of the change may be based on averbal command detected by the user interface 2206 and/or the sensorcomponent 570.

For example, the navigation component 540 may determine that the vehicleshould turn left at the next light (e.g., based on a recommended routeto a destination location). However, the driver sees a fast foodrestaurant on the right side of the street and wants to stop at the fastfood restaurant. Thus, the driver may attempt to override theinstruction and request the vehicle to turn right. The modificationcontroller 2208 may determine if there is a street, or driveway, and mayturn right, as instructed by the driver. If the turn is possible, themodification controller 2208 may provide the instruction to thenavigation component 540, which may recalculate the route.

If the modification controller 2208 determines there is no place to turnright (e.g., no street or driveway at the expected location), themodification controller 2208 may inform the driver of this fact.According to some implementations, more input may be requested from thedriver. In this example, the driver may respond with the restaurant nameand, therefore, the modification controller 2208 may determine that theright turn is located another 500 feet up the road (not directly acrossfrom where the vehicle was going to turn left). Thus, the modificationcontroller 2208 may determine the route change is permissible andnotifies the navigation component 540 to make the alteration to theroute.

In such a manner, changes to a route may be performed on-the-fly or inreal-time based on changing circumstances or other information that maynot be detectable by the system 2200 (e.g., driver is hungry or thirsty,driver sees their friend's vehicle broken down on the side of the roadand wants to help, and so on). Such changes may be implemented by thedriver or other vehicle occupant by interfacing with one or more systemcomponents.

FIG. 25 illustrates example, non-limiting, indications of futuremaneuvers that may be utilized according to one or more embodiments ofthis disclosure. For maneuvers that are further in the future that maybe out of visual range of the vehicle occupants, each maneuver may berepresented by an icon (or avatar). For example, a first examplemaneuver icon 2502 may be an indication that the vehicle will proceed inthe current direction (e.g., straight ahead). A second example maneuvericon 2504 may be an indication that the vehicle will be performing au-turn. A third example maneuver icon 2506 may be an indication that thevehicle will perform a right turn. Additional maneuver icons may beutilized with the disclosed aspects, although not shown or describedherein.

The various maneuver icons may be rendered on a windshield, on a HUDdisplay, on a vehicle device, on a mobile device, or on another devicecommunicatively coupled to the vehicle. Thus, the maneuver icons may bedisplayed spatially and/or temporarily.

According to some implementations, the maneuver icons may be providedwith an optional text description. For example, the second examplemaneuver icon 2504 may include a text description 2508 that indicateswhere the U-turn will take place (e.g., “U-turn at Ellis Street). Inanother example, the third example maneuver icon 2506 may include a textdescription 2510 that indicates where the turn will take place (e.g.,“right on Castro”).

Further, as illustrated in FIG. 25, at substantially the same time asthe maneuver type is rendered, a relevant context (such as laneposition) may be provided. For example, the first example maneuver icon2502 may indicate the vehicle in a right hand lane. In another example,the second example maneuver icon 2504 indicates the vehicle shouldperform the U-turn maneuver from a middle or left hand lane. Further,the third example maneuver icon 2506 indicates the right turn should befrom the curb lane.

FIG. 26 illustrates an example scenario 2600 related to providing visualguides for a current level of steering control according to one or moreembodiments of this disclosure. At the operational level, visualinformation may be provided to indicate a current steering and speedcontrol of the vehicle (which may be the lowest levels of control).

A HUD (or other device) may output perceived lane boundaries and centersteering trajectory. These visual guides may be used to indicate thecurrent level of steering control. For example, boundaries of a lane2602 may be determined by a lane marking manager 2210 that may determinelane boundaries and a center steering trajectory of a lane of traffic.For example, the lane boundaries may be indicated by a first boundaryline 2604 and a second boundary line 2606 of FIG. 26. Further, thecenter steering trajectory may be determined by the lane marking manager2210 to be a center of the lane and displayed as a centerline 2608.Alternatively, the centerline 2608 may be determined by the lane markingmanager 2210 to be a centerline that is located between the first line2604 and the second line 2606, which might not be the center of thetraffic lane.

The lane marking manager 2210 may distinguish the lines such as bymarking the lines in a color that is different than the coloring of theactual (real world) lane markings. For example, the first boundary line2604 and second boundary line 2606 may be a first color and thecenterline 2602 may be a second color. According to otherimplementations, the lines may be distinguished from each other inanother manner, such as dashes, symbols, and so on.

FIG. 27 illustrates an example scenario 2700 related to an incorrectdetermination of a current level of steering control according to one ormore embodiments of this disclosure. Visual guides may be used toindicate the current level of steering control. However, thedetermination of the edge lines and/or centerline may be incorrect.

For example, in the scenario 2600 of FIG. 26, the boundary lines andcenterline were correctly determined by the lane marking manager 2210.However, the system 2200 (e.g., the lane marking manager 2210) mayincorrectly perceive the driving lane. As illustrated in FIG. 27, thereis a first lane 2702, in which the vehicle is currently traversing andwhich is headed in a first direction. There is also a second lane oftraffic 2704, which is headed in a second direction, opposite the firstdirection (e.g., oncoming traffic).

In this case, the lane marking manager 2210 correctly determines thatthe curb edge of the first lane 2702 is a first boundary line 2706.However, in this example, the lane marking manager 2210 incorrectlydetermines that the other side (e.g., curb edge) of the opposite lane2704 of traffic is the other side (e.g., the opposite boundary) of thecurrent lane of traffic 2702. Thus, the lane marking manager incorrectlymarks the curb of the opposite lane of traffic 2704 as the secondboundary line 2708 of the current lane of traffic 2702. Further, thelane marking manager incorrectly places the steering line 2710 betweenthe first boundary line 2706 and the second boundary line 2708 and,therefore, the center of the road between the two lanes of traffic isincorrectly indicated as the steering line (e.g., the vehicle willattempt to stay centered on this steering line). This error may resultin a dangerous situation due the vehicle being instructed to drive downthe middle of the road.

With reference again to FIG. 22, if the vehicle (e.g., the lane markingmanager 2210) has perceived an incorrect driving lane, the driver shouldbe notified (e.g., an alarm should be activated). However, the system2200 is not aware that an error has occurred. If the system 2200 didknow that an error had occurred, the system 2200 would automaticallycorrect the error.

Therefore, according to an implementation, a confidence level determiner2212 may assign a level of confidence to the determination of steeringcontrol. Further, the confidence level determiner 2212 may assignrespective confidence levels to the lane boundaries and the centersteering trajectory.

For example, the lane marking manager 2210 may determine the boundarylines and centerlines, but the system 2200 may determine that there isno oncoming lane of traffic. If the system 2200 has prior knowledgeabout the road being traversed and the knowledge indicates that theoncoming lane of traffic is not contiguous with the current lane (e.g.,it is a divided highway), the confidence level may be high. However, ifthe system 2200 does not have prior knowledge of the road, or theknowledge is from a source whose trust is unknown, the level ofconfidence may be assigned a lower level.

As illustrated in FIG. 28, the confidence level determiner 2212 mayassign different colors (e.g., color cues) to the boundary lines and/orcenter line to illustrate a degree of uncertainty. Thus, the firstboundary line 2706, the second boundary line 2708, and the centerline2710 are output (e.g., by the user interface 2206, the HUD component, oranother system component) such as by showing dashed lines or as redlines. According to some implementations, another type of output may beutilized to draw attention to the level of uncertainty. For example, thelines may be displayed as exclamation points, question marks, geometricshapes, different colors, and so on.

Further, each line may have a different level of confidence. Forexample, the confidence level determiner 2212 may determine that thefirst edge line 2706 is correct based on the current position of thevehicle. Therefore, the first boundary line 2706 is output normally,showing a high degree of certainty that the line is correct. However, itmay be determined by the confidence level determiner 2212 that thesecond boundary line 2708 and the centerline 2210 might not be correctand, therefore, these lines may be output in order to bring attention tothe lower level of certainty.

In an alternate or additional implementation, the confidence leveldeterminer 2212 (or another system 2200 component) may output anotification based on a determination that at least one confidence levelof the respective confidence levels is below a threshold level. In anexample, the notification may be an alarm to advise the occupants of thevehicle about the uncertainty level. For example, a multimodal alarm,such as audio or vibration alerts may be utilized. In another example,the notification may be represented as a visual indicator that isrendered to appear as overlaying real world portions of the lane oftraffic.

According to some implementations, the various alerts may be utilized tolet one or more passengers monitor the potential mismatch betweenperceived and actual road lanes. For example, a passenger may perceive acue or an alert and may notice that the lane markings are not correct.The passenger (or the driver) may inform the system 2200 that the lanemarkings are incorrect. Such information may be conveyed through one ormore sensors components 570, through the user interface 2206, or throughanother system component. Thus, the system 2200 may attempt toautomatically correct the information and/or may receive input from thevehicle occupants for correction of the mismatch.

FIG. 29 illustrates an example, non-limiting, method 2900 may providemulti-level driver monitoring and control according to one or moreembodiments of this disclosure. At 2902, a route may be generatedbetween a first location and a second location. For example, the firstlocation may be determined based on a current location of a vehicle andthe second location may be determined based on a received destinationlocation. For example, a driver of the vehicle may enter an address orother identifier that provides the information related to thedestination location.

At 2904, a map is output on a windshield of the vehicle. The map mayinclude segments of the route. Further, the segments are transparentand, therefore, a view of real world objects through the windshield arenot obstructed by the segments.

At least one segment of the route may be altered, at 2906. The segmentmay be altered based on an indication of a change to the at least onesegment. According to an implementation, altering the segment mayinclude overriding an autonomous steering instruction of the vehicle, at2908.

According to some implementations, altering the at least one segment ofthe route may include detecting a gesture performed by a vehicleoccupant. Further to this implementation, altering the at least onesegment may include interpreting the gesture as a command to change adriving instruction related to the at least one segment.

FIG. 30 illustrates an example, non-limiting method 3000 for navigationmonitoring and control of a vehicle according to one or more embodimentsof this disclosure. At 3002, lane boundaries and a center steeringtrajectory are determined. For example, a first boundary line, a secondboundary line, and a centerline of a current lane of travel may bedetermined.

At 3004, confidence levels may be assigned to the lane boundaries andthe center steering trajectory. For example, a first confidence levelmay be assigned to the first boundary line, a second confidence levelmay be assigned to the second boundary line, and a third confidencelevel may be assigned to the centerline.

Representations of the lane boundaries and the center steeringtrajectory are output, at 3006. According to an implementation,representations of the first boundary line, the second boundary line,and the centerline are based, at least in part, on the first confidencelevel, the second confidence level, and the third confidence level. Forexample, if one confidence level of the respective confidence levels isbelow a threshold amount, the representation associated with thatconfidence level may be displayed differently than the otherrepresentations.

As discussed herein, according to one aspect, a vehicle heads-up displaydevice for displaying graphic elements in view of a driver of a vehicleincludes a first projector and a first actuator. The first projector mayproject a first graphic element on a first focal plane in view of thedriver. The first focal plane may be oriented substantiallyperpendicularly to a line-of-sight of the driver and a distance awayfrom the vehicle. The first projector may be mounted on the firstactuator. The first actuator may linearly move the first projector.Linearly moving the first projector may cause the first focal plane ofthe first graphic element to move in a direction of the line-of-sight ofthe driver.

According to another aspect, a vehicular heads-up display systemincludes a vehicle heads-up display device and a controller. The vehicleheads-up display device displays graphic elements in view of a driver ofa vehicle, and includes a first projector and a second projector. Thefirst projector may project a first graphic element on a first focalplane in view of the driver. The first focal plane may be orientedsubstantially perpendicularly to a line-of-sight of the driver. Thefirst projector may move the first focal plane in a direction of theline-of-sight of the driver. The second projector may project a secondgraphic element on a second focal plane in view of the driver. Thesecond focal plane may be static and oriented substantially parallel toa ground surface. The controller may communicate with one or moreassociated vehicle control systems and to control the vehicle heads-updisplay device to display the first and second graphic elements based oncommunication with one or more of the associated vehicle controlsystems.

According to yet another aspect, a method for presenting augmentedreality graphic elements in a vehicle heads-up display includesprojecting a first graphic element on a first focal plane in view of adriver, and a second graphic element on a second focal plane in view ofthe driver. The first focal plane may be oriented substantiallyperpendicularly to a line-of-sight of the driver, and the second focalplane may be static and oriented substantially parallel to a groundsurface. The method may include moving or adjusting the first focalplane in a direction of the line-of-sight of the driver.

One or more embodiments of techniques or systems for 3-dimensional (3-D)navigation are provided herein. For example, a system for 3-D navigationmay project a graphic element or avatar that appears to move in view ofan occupant of a vehicle. In one or more embodiments, a heads-up displaycomponent (HUD) component may project the graphic element or avatar onone or more focal planes in an environment surrounding a vehicle. TheHUD component may project graphic elements or avatars at adjustabledistances or adjustable focal planes to provide an occupant of a vehiclewith the perception that an avatar or graphic element is moving, flying,animated, etc.

As an example, the HUD component may ‘animate’ or provide movement foran avatar by sequentially projecting the avatar on one or more differentfocal planes. Projection on to these focal planes may be achievedutilizing an actuator to move a projector of the HUD component, forexample. As a result, depth cues such as accommodation and vergenceassociated with a graphic element or avatar are generally preserved.When a route is generated from a first location to a second location,the HUD component may generate one or more graphic elements for a driveror occupant of a vehicle to ‘follow’. The HUD component may project ontomultiple focal planes or move projected graphic elements from one focalplane to another, therefore, graphic elements or projected images mayappear much more ‘real’, similar to an image seen in a mirror.

When an occupant of a vehicle requests navigation guidance, a graphicelement, such as an avatar, may be provided. The avatar may appear tomove, glide, fly, etc. in front of the vehicle, similar to what anoccupant or driver would see if they were following a friend's vehicle,for example. Additionally, the avatar might appear to navigate aroundobstructions, obstacles, pedestrians, debris, potholes, etc. as a realvehicle would. In one or more embodiments, the avatar might ‘drive’,move, appear to move, etc. according to real-time traffic. For example,if a route takes a driver or a vehicle across train tracks, the avatarmay stop at the train tracks when a train is crossing. As anotherexample, the avatar may change lanes in a manner such that the avatardoes not appear to ‘hit’ another vehicle or otherwise interfere withtraffic.

In one or more embodiments, a sensor component may track or detect oneor more objects or information associated with the environment in whicha vehicle is travelling. This information may be passed to the vehiclecontrol component. The vehicle control component may aggregate, collect,or receive information about the environment around the vehicle fromdifferent sources. A controller component may utilize this informationto make decisions regarding whether or not to project a graphic elementat a location or on a focal plane and attributes or models to utilizewhen rendering the graphic element. A HUD component may render orproject the graphic element based on the controller component decision.

For example, if a sensor component detects a pedestrian in a crosswalkacross a road segment along a route which involves having the vehicleturn onto that road segment, the controller component may order orcommand the HUD component to project a graphic element on or around thepedestrian to alert a driver or occupant of the vehicle of the presenceof the pedestrian. Further, the sensor component may actively track thepedestrian such that the HUD component may project graphic elements,such as pointers, which follow or track the pedestrian in real-time ashe or she crosses the crosswalk. Additionally, the controller componentmay identify or recognize when the pedestrian has finished crossing orsteps onto a sidewalk and communicate to the HUD component to ceaseprojecting graphic elements for the pedestrian.

A system for 3-D navigation may enable identification or projection ofearly warning graphic elements. For example, if a driving is turningfrom a first road segment to a second road segment (e.g., making a leftturn or a right turn from one road to another), information received bythe vehicle control component, such as environment information ortraffic condition information may be utilized by the controllercomponent to make a determination that an early warning graphic elementshould be rendered for an occupant or driver of a vehicle. Thecontroller component may order or command the HUD component to render agraphic element which points to an object (e.g., a hazard, roadcondition, etc.) which may not necessarily be in view of the driver.When looking out of the windshield of the vehicle, a driver may not seea hazard on his or her left, for example. However, the HUD component mayproject a graphic element which directs the driver's attention to his orher left, (e.g., off the windshield or outside of the view of thedriver). Safety may be promoted by providing drivers or occupants of avehicle with advance notice of objects within an environment.

Additionally, the HUD component may project or render details pertainingto objects within the environment, such as a name of a business, addressmarkers, final destination markers, parking graphic elements, reviews,etc.

One or more embodiments of techniques or systems for 3-dimensional (3-D)navigation are provided herein. A heads-up display (HUD) component mayproject, render, display, or present graphic elements on focal planesaround an environment surrounding a vehicle. The HUD component may causethese graphic elements to appear volumetric or 3-D by moving oradjusting a distance between a focal plane and the vehicle. Objectswithin the environment may be tracked, identified, and correspondinggraphic elements may be projected on, near, or around respectiveobjects. For example, the HUD component may project graphic elements orpointers on pedestrians such that it may alert the driver or operator ofthe vehicle of their presence. These pointers may stay ‘stuck’ on thepedestrian as he or she is walking within the environment. Metadataassociated with objects may be presented, such as address information,ratings, telephone numbers, logos, etc.

Still another embodiment involves a computer-readable medium includingprocessor-executable instructions that implement one or more embodimentsof the techniques presented herein. An embodiment of a computer-readablemedium or a computer-readable device that is devised in these ways isillustrated in FIG. 31, wherein an implementation 3100 includes acomputer-readable medium 3108, such as a CD-R, DVD-R, flash drive, aplatter of a hard disk drive, etc., on which is encodedcomputer-readable data 3106. This computer-readable data 3106, such asbinary data including a plurality of zeroes or ones as shown in 3106, inturn includes a set of computer instructions 3104 that operate accordingto one or more of the principles set forth herein. In one suchembodiment 3100, the processor-executable computer instructions 3104 mayperform a method 3102, such as the method 2900 of FIG. 29 or the method3000 of FIG. 30. In another embodiment, the processor-executableinstructions 3104 may implement a system, such as the system 2200 ofFIG. 22. Many such computer-readable media are devised by those ofordinary skill in the art that may operate in accordance with thetechniques presented herein.

As used in this application, the terms “component,” “module,” “system,”“interface,” and the like are generally intended to refer to acomputer-related entity, either hardware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,or a computer. By way of illustration, both an application running on acontroller and the controller may be a component. One or more componentsresiding within a process or thread of execution and a component may belocalized on one computer or distributed between two or more computers.

Further, the claimed subject matter is implemented as a method,apparatus, or article of manufacture using standard programming orengineering techniques to produce software, firmware, hardware, or anycombination thereof to control a computer to implement the disclosedsubject matter. The term “article of manufacture” as used herein isintended to encompass a computer program accessible from anycomputer-readable device, carrier, or media. Of course, manymodifications may be made to this configuration without departing fromthe scope or spirit of the claimed subject matter.

FIG. 32 and the following discussion provide a description of a suitablecomputing environment to implement embodiments of one or more of theprovisions set forth herein. The operating environment of FIG. 32 ismerely one example of a suitable operating environment and is notintended to suggest any limitation as to the scope of use orfunctionality of the operating environment. Example computing devicesinclude, but are not limited to, personal computers, server computers,hand-held or laptop devices, mobile devices, such as mobile phones,Personal Digital Assistants (PDAs), media players, and the like,multiprocessor systems, consumer electronics, mini computers, mainframecomputers, distributed computing environments that include any of thedisclosed systems or devices, and the like.

Generally, embodiments are described in the general context of “computerreadable instructions” being executed by one or more computing devices.Computer readable instructions are distributed via computer readablemedia as will be discussed below. Computer readable instructions areimplemented as program modules, such as functions, objects, ApplicationProgramming Interfaces (APIs), data structures, and the like, thatperform one or more tasks or implement one or more abstract data types.Typically, the functionality of the computer readable instructions arecombined or distributed as desired in various environments.

FIG. 32 illustrates a system 3200 including a computing device 3212 thatimplements one or more embodiments provided herein. In oneconfiguration, computing device 3212 includes one or more processingunits 3216 and memory 3218. Depending on the exact configuration andtype of computing device, memory 3218 may be volatile, such as RAM,non-volatile, such as ROM, flash memory, etc., or a combination of thetwo. This configuration is illustrated in FIG. 32 by dashed line 3214.

In other embodiments, device 3212 includes additional features orfunctionality. For example, device 3212 may include additional storagesuch as removable storage or non-removable storage, including, but notlimited to, magnetic storage, optical storage, and the like. Suchadditional storage is illustrated in FIG. 32 by storage 3220. In one ormore embodiments, computer readable instructions to implement one ormore embodiments provided herein are in storage 3220. Storage 3220 maystore other computer readable instructions to implement an operatingsystem, an application program, and the like. Computer readableinstructions are loaded in memory 3218 for execution by processing unit3216, for example.

The term “computer readable media” as used herein includes computerstorage media. Computer storage media includes volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions or other data. Memory 3218 and storage 3220 are examples ofcomputer storage media. Computer storage media includes, but is notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, Digital Versatile Disks (DVDs) or other optical storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or any other medium which may be used to storethe desired information and which may be accessed by device 3212. Anysuch computer storage media is part of device 3212.

The term “computer readable media” includes communication media.Communication media typically embodies computer readable instructions orother data in a “modulated data signal” such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” includes a signal that has one or more ofits characteristics set or changed in such a manner as to encodeinformation in the signal.

Device 3212 includes input device(s) 3224 such as keyboard, mouse, pen,voice input device, touch input device, infrared cameras, video inputdevices, or any other input device. Output device(s) 3222 such as one ormore displays, speakers, printers, or any other output device may beincluded with device 3212. Input device(s) 3224 and output device(s)3222 are connected to device 3212 via a wired connection, wirelessconnection, or any combination thereof. In one or more embodiments, aninput device or an output device from another computing device are usedas input device(s) 3224 or output device(s) 3222 for computing device3212. Device 3212 may include communication connection(s) 3226 tofacilitate communications with one or more other devices.

According to one or more aspects, a system for 3-dimensional (3-D)navigation is provided, including a navigation component that receivesan origin location and a destination location. The navigation componentmay be associated with a vehicle and may generate a route from theorigin location to the destination location. One or more portions of theroute may include one or more navigation instructions associated withone or more road segments or one or more intersections of the roadsegments. The system may include a heads-up display (HUD) component thatmay project one or more graphic elements on one or more focal planesaround an environment surrounding the vehicle. The HUD component mayproject one or more of the graphic elements in view of an occupant ofthe vehicle based on the route. The system may include a controllercomponent that may adjust a distance between one or more of the focalplanes of one or more of the graphic elements and the vehicle based onone or more road conditions associated with one or more portions of theroute and a current position of the vehicle.

In one or more embodiments, the controller component may adjust a targetposition for one or more of the graphic elements based on one or more ofthe road conditions and the current position of the vehicle. The systemmay include a vehicle control component may receive one or more of theroad conditions. Additionally, the system may include a sensor componentmay detect one or more of the road conditions. A road condition of theone or more road conditions may include traffic information of one ormore of the road segments or speed limit information associated with oneor more of the road segments. Additionally, road conditions may includean obstruction, an obstacle, a pedestrian, debris, or a pothole, forexample.

The system may include a depth map component may build a depth map ofthe environment surrounding the vehicle. The HUD component may projectone or more of the graphic elements based on the depth map of theenvironment. The depth map component may build the depth map based ondepth information. In one or more embodiments, the system may include asensor component may detect depth information from the environmentsurrounding the vehicle. The depth map component may receive the depthmap based on a telematics channel. The system may include a depthbuffering component may enable or disable rendering of one or moreportions of one or more of the graphic elements based on the depth map.

The HUD component may project one or more graphic elements as a movingavatar or as a placeholder, such as a flag pole, marker, identifier,etc.

According to one or more aspects, a method for 3-dimensional (3-D)navigation is provided, including generating a route from an originlocation to a destination location for a vehicle. One or more portionsof the route may include one or more navigation instructions associatedwith one or more road segments or one or more intersections of the roadsegments. The method may include projecting one or more graphic elementson one or more focal planes around an environment surrounding thevehicle. One or more of the graphic elements may be projected in view ofan occupant of the vehicle based on the route. The method may includeadjusting a distance between one or more of the focal planes of one ormore of the graphic elements and the vehicle based on one or more roadconditions associated with one or more portions of the route and acurrent position of the vehicle. One or more portions of the method maybe implemented via a processing unit.

The method may include adjusting a target position for one or more ofthe graphic elements based on one or more of the road conditions and thecurrent position of the vehicle. The method may include receiving ordetecting one or more of the road conditions. A road condition of theone or more road conditions may include traffic information of one ormore of the road segments, speed limit information associated with oneor more of the road segments, an obstruction, an obstacle, a pedestrian,debris, or a pothole.

The method may include building a depth map of the environmentsurrounding the vehicle, projecting one or more of the graphic elementsbased on the depth map of the environment, detecting depth informationfrom the environment surrounding the vehicle, building the depth mapbased on the detected depth information, enabling or disabling renderingof one or more portions of one or more of the graphic elements based onthe depth map, among other things.

According to one or more aspects, a computer-readable storage mediumincluding computer-executable instructions, which when executed via aprocessing unit on a computer performs acts, including generating aroute from an origin location to a destination location for a vehicle,wherein one or more portions of the route include one or more navigationinstructions associated with one or more road segments or one or moreintersections of the road segments, projecting one or more graphicelements on one or more focal planes around an environment surroundingthe vehicle, wherein one or more of the graphic elements are projectedin view of an occupant of the vehicle based on the route, or adjusting adistance between one or more of the focal planes of one or more of thegraphic elements and the vehicle based on one or more road conditionsassociated with one or more portions of the route and a current positionof the vehicle.

In one or more embodiments, projecting one or more of the graphicelements utilizes rastor based graphics. Additionally, one or more ofthe embodiments may include providing one or more of the navigationinstructions via projecting one or more of the graphic elements as amoving avatar or animating the moving avatar by sequentially projectingthe moving avatar on one or more different focal planes.

According to one or more aspects, a system for 3-dimensional (3-D)navigation is provided, including a sensor component, a heads-up display(HUD) component, and a controller component. The sensor component maytrack one or more objects in an environment surrounding a vehicle andone or more corresponding coordinates for respective objects relative tothe vehicle. The HUD component may project, render, present, or displayone or more graphic elements on one or more focal planes correspondingto one or more of the objects, wherein one or more of the graphicelements are projected in view of an occupant of the vehicle. Thecontroller component may calculate one or more updated focal planes forone or more of the graphic elements based on one or more of thecoordinates for respective objects.

In one or more embodiments, the HUD component may project one or more ofthe graphic elements on one or more of the updated focal planes for oneor more of the objects. Additionally, the HUD component may ceaseprojecting one or more of the graphic elements on one or more of thefocal planes upon projecting on one or more of the updated focal planes.One or more of the graphic elements may include a pointer havinginformation associated with one or more of the objects on which one ormore of the graphic elements is projected. One or more of the objectsmay be an obstruction, an obstacle, a pedestrian, a construction zone, alandmark, a building, a business, or a parking spot.

The system may include a vehicle control component determining a size, ashape, a model, a color, or one or more attributes for one or more ofthe graphic elements. The vehicle control component may manage overlapbetween two or more of the graphic elements. The system may include anavigation component generating a route from an origin location to adestination location and a vehicle control component receivinginformation associated with one or more of the objects along the route,wherein one or more of the objects are businesses. The HUD component mayproject one or more of the graphic elements as a logo associated of oneor more of the businesses. In one or more embodiments, the sensorcomponent may include a navigation component or utilize telematics.

According to one or more aspects, a system for 3-dimensional (3-D)navigation is provided, including a navigation component, a controllercomponent, and a heads-up display (HUD) component. The navigationcomponent may receive metadata associated with one or more objects in anenvironment surrounding a vehicle, a layout for respective objectswithin the environment, and a current location of the vehicle relativeto the layout. The controller component may calculate one or more focalplanes for one or more graphic elements based on the layout of one ormore of the objects and the current location of the vehicle. The HUDcomponent may project, render, present, or display one or more of thegraphic elements on one or more of the focal planes corresponding to oneor more of the objects, wherein one or more of the graphic elements areprojected in view of an occupant of the vehicle.

One or more of the graphic elements may include a pointer presenting atleast a portion of the metadata associated with one or more of theobjects on which one or more of the graphic elements is projected. Themetadata may include a telephone number, an address, a rating, a name ofa business, hours of operation, or a status associated with an object.The HUD component may project one or more of the graphic elements basedon the layout of one or more of the objects and the current location ofthe vehicle. Additionally, the HUD component may shift from projecting alayout view or a birds-eye-view to projecting a first-person view or athird-person view.

The system may include a sensor component receiving a query from theoccupant of the vehicle associated with one or more of the objects,wherein the HUD component projects one or more portions of correspondingmetadata for one or more of the objects associated with the query inresponse to the query. The navigation component may generate a routefrom an origin location to a destination location, wherein the HUDcomponent renders one or more of the graphic elements as an avatar whichleads the occupant along the route. The controller component maycalculate a pitch angle, roll angle, yaw angle, or velocity for theavatar (or other graphic elements). The HUD component may present,display, or render one or more of the graphic elements as a road namebased on the metadata and the current location of the vehicle.

According to one or more aspects, a method for 3-dimensional (3-D)navigation is provided, including tracking one or more objects in anenvironment surrounding a vehicle and determining one or morecorresponding coordinates for respective objects relative to thevehicle, calculating one or more focal planes for one or more graphicelements based on one or more of the coordinates for respective objects,and rendering one or more of the graphic elements on one or more of thefocal planes corresponding to one or more of the objects, wherein one ormore of the graphic elements are projected in view of an occupant of thevehicle. The tracking, the calculating, or the rendering may beimplemented via a processing unit.

Although the subject matter has been described in language specific tostructural features or methodological acts, it is to be understood thatthe subject matter of the appended claims is not necessarily limited tothe specific features or acts described herein. Rather, the specificfeatures and acts described herein are disclosed as example embodiments.

Various operations of embodiments are provided herein. The order inwhich the various operations are described should not be construed as toimply that these operations are necessarily order dependent. Alternativeordering will be appreciated based on this description. Further, not alloperations may necessarily be present in each embodiment providedherein.

As used in this application, “or” is intended to mean an inclusive “or”rather than an exclusive “or.” In addition, “a” and “an” as used in thisapplication are generally construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Additionally, at least one of A and B and/or the like generallymeans A or B or both A and B. Further, to the extent that “includes”,“having”, “has”, “with”, or variants thereof are used in either thedetailed description or the claims, such terms are intended to beinclusive in a manner similar to the term “comprising”.

Further, unless specified otherwise, “first”, “second”, or the like arenot intended to imply a temporal aspect, a spatial aspect, an ordering,etc. Rather, such terms are merely used as identifiers, names, etc. forfeatures, elements, items, etc. For example, a first channel and asecond channel generally correspond to channel A and channel B or twodifferent or two identical channels or the same channel.

Although the disclosure has been shown and described with respect to oneor more implementations, equivalent alterations and modifications willoccur based on a reading and understanding of this specification and theannexed drawings. The disclosure includes all such modifications andalterations and is limited only by the scope of the following claims.

What is claimed is:
 1. A system for multi-level navigation monitoringand control, comprising: a memory; and a processor that executes thefollowing computer executable components stored in the memory: anavigation manager generating a route between a first location and asecond location; a map displayer outputting a map that includes theroute and is indicative of distances of one or more surfaces, objects,obstructions, or geometries in an area around a vehicle, wherein the mapdisplayer outputs the route to indicate a path travelled and a futurepath expected to be travelled, each graphically distinguishable from oneanother, wherein the map displayer outputs the map on a windshield ofthe vehicle, wherein the map is transparent and does not obstruct a viewof real world objects in a vehicle environment; a modificationcontroller altering at least a portion of the route based on anindication of a change received at a user interface or a sensor; a lanemarking manager determining one or more lane boundaries and a centersteering trajectory of a lane of traffic; and a confidence leveldeterminer assigning respective confidence levels to the lane boundariesand the center steering trajectory based on a prior system knowledge ofconditions of a road being traversed, respective confidence levelsindicative of confidence that the lane boundaries are correct, whereinthe navigation manager regenerates the route between the first locationand the second location as a function of the indication of the changeand the map displayer outputs a revised map based on the regeneratedroute.
 2. The system of claim 1, wherein the indication of the change isbased on a gesture detected by the user interface or the sensor.
 3. Thesystem of claim 1, wherein the indication of the change is based on averbal command detected by the user interface or the sensor.
 4. Thesystem of claim 1, wherein the confidence level determiner is configuredto output outputting a notification based on a determination that atleast one confidence level of the respective confidence levels is belowa threshold level.
 5. The system of claim 4, wherein the notification isa multimodal alarm.
 6. The system of claim 4, wherein the notificationis represented as a visual indicator that is rendered to appear asoverlaying real world portions of the lane of traffic.
 7. The system ofclaim 1, wherein the indication of the change is received from apassenger in the vehicle.
 8. A method for multi-level navigationmonitoring and control, comprising: generating a route between a firstlocation and a second location based on a determination of a currentlocation of a vehicle and a received destination location; outputting amap on a windshield of the vehicle, the map comprises segments of theroute and are transparent, wherein a view of real world objects throughthe windshield are not obstructed by the segments, wherein theoutputting the route includes indicating a path travelled and a futurepath expected to be travelled, each graphically distinguishable from oneanother; altering at least one segment of the route based on anindication of a change to the at least one segment, the alteringcomprising overriding an autonomous steering instruction of the vehicle;determining one or more lane boundaries and a center steering trajectoryof a lane of traffic; overlaying the lane boundaries and the centersteering trajectory on a real world view of the lane of traffic;assigning respective confidence levels to the lane boundaries and thecenter steering trajectory based on a prior system knowledge ofconditions of a road being traversed, respective confidence levelsindicative of confidence that the lane boundaries are correct; andaltering an output of the overlaid lane boundaries and the centersteering trajectory based on the assigned confidence levels.
 9. Themethod of claim 8, wherein the altering at least one segment of theroute comprises: detecting a gesture performed by a vehicle occupant;and interpreting the gesture as a command to change a drivinginstruction related to the at least one segment.
 10. The method of claim8, further comprising: determining at least one confidence level of therespective confidence levels is below a threshold amount; and outputtinga multimodal alarm based on the determining.
 11. A system for navigationmonitoring and control of a vehicle, comprising: a memory; and aprocessor that executes the following computer executable componentsstored in the memory: a lane marking manager determining a firstboundary line, a second boundary line, and a centerline of a currentlane of travel; a confidence level determiner assigning a firstconfidence level to the first boundary line, a second confidence levelto the second boundary line, and a third confidence level to thecenterline based on a prior system knowledge of conditions of a roadbeing traversed, respective confidence levels indicative of confidencethat the lane boundaries are correct; and a user interface outputtingrepresentations of the first boundary line, the second boundary line,and the centerline based on the first confidence level, the secondconfidence level, and the third confidence level.
 12. The system ofclaim 11, wherein the user interface is configured to output outputtingthe representations in a first format based on the respective confidencelevels conforming to a predetermined threshold value, and outputtingoutput the representations in a second format based on the respectiveconfidence levels being below the predetermined threshold value.
 13. Thesystem of claim 11, wherein the user interface outputting is configuredto output an audible alarm based on a determination that at least one ofthe first confidence level, the second confidence level, and the thirdconfidence level are below a threshold amount.
 14. The system of claim11, further comprising at least one sensor receiving a modification toat least one of the first boundary line, the second boundary line, orthe centerline, the confidence level determiner assigning anotherconfidence level based on the modification, and the user interfacechanging the respective representation based on the modification and theother confidence level.
 15. The system of claim 11, wherein the userinterface is configured to output outputting the representations on awindshield, wherein the representations are rendered to overlayassociated real world portions of the current lane of travel.